What is RDMA?

Remote Direct Memory Access – RDMA – is a data transfer method supporting greater throughput compared to traditional transfer methods. While it’s been used in data centers and by networking innovators for some years, it’s only recently been made available for machine vision.

In a phrase: RDMA is a method of transferring data directly between a device and a designated area of computer memory with little involvement from the operating system and CPU.

Some image transfer context to motivate the need for RDMA

Cameras doing real time machine vision don’t send entire images in a single file – it’s not like jpg images from a smartphone. Rather, sensor data is read out in pixel values conveyed in bytes in turn assembled into packets. The operating system, CPU, and networking protocol move the packets and assemble the image in the PC host.

At “conventional” GigE speeds like 1GigE, 5GigE, and 10GigE, a regular NIC and your PC host processor can typically keep up with the framerates flowing from the camera to the PC and vision application software.

But at 25GigE and especially 100GigE, which you may need for your high framerate application, the CPU might get so overwhelmed with data copy and network protocol management that not enough CPU cycles remain for image processing per se. Which in turn risks frame loss and latency bottlenecks.

An analogy

Suppose a package has to get to a specific recipient’s desk in a mid-town office building or on a large industrial campus with multiple buildings and departments. One method would be to get it from the external shipper to a receiving department. Who then transport it to the department level. Who then get it to the business unit. And finally to the intended individual. It works. But it’s a lot of handling, each of which has a cost in labor and time and hence efficiency. That’s equivalent to the traditional network packetization and delivery approach.

The RDMA-equivalent approach would be to authorize the external delivery service (once suitably credentialed) to deliver the parcel directly to the intended desktop, without all the intermediate handling steps.

RDMA at a glance

Recall that GigE Vision utilizes GVSP – GigE Vision Streaming Protocol. Which in turn depends on UDP – User Datagram Protocol. Which is built on top of network layer IP – Internet Protocol. GigE Vision 3.0 added an RDMA-based streaming option, GVRSP, for applications that need higher data rates and less CPU overhead.

Translating that into conventional GigE data transfer, there are typically the four layers shown in the left-hand side of the diagram below: Application, Socket, Transfer Protocol, and Driver level. All those are abstracted away from the user/programmer by your GigE Vision compliant camera, NIC, and SDK – but the layers are all in there. And the successive handoffs between the layers, at both the transmission and receiving side, require communications handoffs and data buffering. Which incurs performance costs in time as well as processor load.

Reduce number of handling instances with RDMA increases throughput – Courtesy Allied Vision

Reduced buffering and handoffs with RDMA

Now look at the right-hand side of the diagram above, utilizing RDMA. Three handoff and two buffering layers are bypassed, yielding substantial throughput gains. Just as with traditional GigE Vision, the user/programmer does not have to manage data transport. Your device manufacturers and SDK provider take of that for you. But you get to harvest the performance gains.

Zero-copy

As you get into RDMA, you may see the “zero-copy” characteristic touted as a benefit. Don’t be confused by the term “copy”. It doesn’t mean conventional networking makes duplicate copies. Merely that conventional networking has to move a packet from one buffer to another, so it copies to the new and releases the old. It’s the “reduced handling instances” concept illustrated in the diagram above.

Key benefits

Higher sustained throughput – host PC doesn’t get bogged down

Lower CPU utilization – frees up capacity for image processing

Lower latency and jitter – image delivery faster and regularized

Improved stability – avoids dropped packets or frames

Putting the pieces together

If your application requires the performance delivered by RDMA, despite being a GigE protocol, one uses fiber cables instead of conventional Cat6 ethernet cables. One must convert between copper and glass fiber transmission modes at both the transmission and receiving ends. Hence the transceivers as shown in the illustration below.

Key components for a machine vision RDMA setup – Courtesy Allied Vision

Click to contact
Give us some brief idea of your application and we will contact you to discuss camera and cabling options.

How long a fiber run do you need?

There are transceivers designed for several different throughput and cable length requirements. Costs scale accordingly. But innovative machine vision solutions that bring competitive advantage can drive revenue or reduce other costs, so it’s partly about engineering possibilities and partly about cost : benefit calculation.

Transceivers for different bandwidth and cable length requirements – Courtesy Allied Vision

Note: the multi-km cable lengths are a benefit of fiber optics, not of RDMA per se. It’s a happy coincidence that RDMA performance at 25 and 100GigE is best supported by fiber optics; and that once a signal is on fiber, and can be reliably transmitted a long distance.


Do you need 25 or 100 GigE? Fiber optic cables? RDMA? Start with required application outcomes in terms of image characteristics and framerate, and work back to sensor, camera, interface, and techniques. We do that.

1st Vision’s sales engineers have over 100 years of combined experience to assist in your camera and components selection.  With a large portfolio of cameraslensescablesNIC cards and industrial computers, we can provide a full vision solution!

About you: We want to hear from you!  We’ve built our brand on our know-how and like to educate the marketplace on imaging technology topics…  What would you like to hear about?… Drop a line to info@1stvision.com with what topics you’d like to know more about.

#RDMA

#Remotedirectmemoryaccess

HDR in machine vision – solving bright and dark imaging challenges

HDR image

High dynamic range (HDR) isn’t new. It’s frequently mentioned. And offered on-camera, or via software or FPGA. Is it just a marketing term, or a real benefit? If your application’s scenes contain both bright and dark regions, HDR can absolutely deliver benefits.

Consider the three images shown below:

Courtesy JAI

Neither the “slow shutter” image nor the “fast shutter” image is optimal. The former is over-saturated – one can’t even find the many windows in the central building. The fast shutter image is of course too dark, essentially losing the arch and the flagpole. While this scene is more from the realm of “photography” than “machine vision”, the concepts are the same.

Clearly the best image is the HDR image – the lighter areas are revealed in nuanced detail, but so too the unlit trees and gray windows are clear in their own degrees of black and gray, and everything in between. How is this achieved?

Contact us

What is HDR?

Let’s unpack the acronym, starting with DR for dynamic range. DR is the ratio between the largest and smallest measurable values, for the quantity being measured. For machine vision, it’s light intensity that’s being quantified.

Generally speaking, a larger dynamic range is preferrable to a small one, as the nuanced differences of a relatively larger dynamic range may be required for effective image processing. Take edge-detection, a common machine vision requirement for many applications. The edge may only become apparent, under given lighting conditions and resolution, when the saturation of pixels in a given region are consistently lower to one side and consistently higher to the other side of the “emergent” edge. With sufficient dynamic range, calculated confidence grows, while poor dynamic range may fail to reveal an edge at all.

Ways to create a composite HDR image

One way to create an HDR image is with two exposures and an algorithm for creating the composite. The shorter exposure captures the more brightly lit or highly reflective surfaces, while the remaining regions remain unsaturated or only slightly registering. A longer exposure oversaturates the lighter targets, but reveal nuanced variation in the previously unrevealed details.

In fact one does the longer exposure first, such that the darker portions of the scene produce a variance of non-zero values – i.e. a dynamic range across the darker regions.

Then for the shorter exposure, use the brightest non-saturated pixels from the first exposure as a reference to generate small non-zero values as a control on the short exposure, creating a calculated point of overlap. That way many pixels that were oversaturated on the long exposure are only slightly to moderated saturated on the short-exposure, for a nuanced spread of values across the corresponding pixels.

The blending algorithm compares the two images, pixel for pixel, with the overlap point as a reference. Saturated pixels in the first image are replaced with the corresponding non-saturated pixel values from the second image.

While the two-exposure approach described above is easy to understand, there’s clearly a time-cost in taking two successive exposures, reading them both out to the PC host, and doing the image processing. For certain applications, that may be acceptable. For others, especially with motion involved, or desired high cycle counts, one might hope for a faster approach.

Another way: multi-slope pixel generation on CMOS sensors

The rise of CMOS sensors and their transistor-based pixel architecture enables on-sensor functionality that convenient supports the generation of HDR images. This may be achieved by resetting pixels approaching saturation, prior to end of exposure, so those pixels have an opportunity to be filled from a range of values instead of maxing out had the reset not occurred.

Consider the follow two diagrams, and the supporting discussion below:

If many pixels fill before the end of the exposure, a lot of the image may be oversaturated, even though the darker regions need a longer exposure to become meaningfully non-zero. Courtesy JAI.

But thanks to CMOS transistors at each pixel position, the sensor can be programmed to monitor saturation values, and to reset pixels approaching saturation to “partial fill” levels that allow additional fill for the remainder of the exposure.

Courtesy JAI

It gets even better

Above was “intro level” HDR, concepts and techniques that provide the foundation. Meanwhile innovators keep taking it to the next level.

For example, Sony now offers Quad HDR on their IMX900 sensor, available in the IDS uEye low-cost cameras. Getting the dark sections sufficiently saturated while not oversaturating the brighter regions is really evident with Quad HDR below.

Quad HDR generates a balanced image – Courtesy IDS

In the video below, you may jump to position 1 minute 42 seconds for more on Quad HDR:


Even more on HDR:

If you’d like to read a more in-depth treatment on HDR, including more example images, supporting arithmetic and graphical rational, download our whitepaper on High Dynamic Range Imaging.

Or perhaps you have an application with known nuanced dark regions as well as variation in the saturated areas, for which HDR may add value. Should you do it on-camera/sensor? In an FPGA/frame-grabber? On the PC host? Use lighting techniques to avoid needing HDR altogether? There are a number of different ways to achieve optimal image outcomes, but HDR is certainly a valuable technique for some applications.

Call us at 978-474-0044, and let us guide you to a best-fit solution.

1st Vision’s sales engineers have over 100 years of combined experience to assist in your camera and components selection.  With a large portfolio of cameraslensescablesNIC cards and industrial computers, we can provide a full vision solution!

About you: We want to hear from you!  We’ve built our brand on our know-how and like to educate the marketplace on imaging technology topics…  What would you like to hear about?… Drop a line to info@1stvision.com with what topics you’d like to know more about.

#HDR

#Highdynamicrange

Collimated lighting important with telecentric lens

LTCLHP Collimated Light – Courtesy Opto Engineering

Machine vision practitioners, regardless of application or lens type, know that contrast is essential. Without sharp definition, features cannot be detected effectively.

When using a telecentric lens for precision optical 2-D measurements, ideally one should also use collimated lighting. Per the old adage about a chain being only as good as its weakest link, why invest in great lensing and then cut corners on lighting?

contact us

WITH collimated light expect high edge definition:

The cost of the light typically pays for itself relative to quality outcomes. Below see red-framed enlargements of the same region of a part being imaged by the same telecentric lens.

The left-hand image was taken with a conventional backlight – note how the light wraps around the edge, creating “confusion” and imprecision due to refracted light coming from all angles.

The right-hand image was obtained with a collimated backlight – with excellent edge definition.

Conventional backlight (left) vs. collimated backlight (right) – Courtesy Opto Engineering.

It all comes down to resolution

While telecentric imaging is a high-performance subset of the larger machine vision field in general, the same principles of resolution apply. It takes several pixels to confidently resolve any given feature – such as an edge – so any “gray areas” induced by lower quality lighting or optics would drag down system performance. See our blog and knowledge-base coverage of resolution for more details.

Collimated lighting in more detail

Above we see the results of using “diffuse” vs. “collimated” light sources, which are compelling. But what is a collimated light and how does it work so effectively?

UNLIKE a diffuse backlight, whose rays emanate towards the object at angles ranging from 0 to almost 180°, a collimated backlight sends rays with only very small deviations from perfectly parallel. Since parallel rays are also all that the telecentric lens receives and transmits on to the camera sensor, stray rays are mitigated and essentially eliminated.

The result is a high-contrast image which is easier to process with high-reliability. Furthermore, shutter speeds are typically faster, achieving necessary saturation more quickly, thereby shortening cycle times and increasing overall throughput.

Many lights to choose from:

The video below shows a range of light types and models, including clearly labeled direct, diffuse, and collimated lights.

Several light types – including clearly labeled collimated lights

[Optional] Telecentric concepts overview

Below please compare the diagrams that show how light rays travel from the target position on the left, through the respective lenses, and on to the sensor position on the far right.

A telecentric lens is designed to insure that the chief rays remain parallel to the optical axis. The key benefit is that (when properly focused and aligned) the system is invariant to the distance of the object from the lens. This effectively ignores light rays coming from other angles of incidence, and thereby supports precise optical measurement systems – a branch of metrology.

If you’d like to go deeper on telecentrics, see the following two resources:

Telecentric concepts presented as a short blog.

Alternatively as a more comprehensive Powerpoint from our KnowledgeBase.

Video: Selecting a telecentric lens:

Call us at 978-474-0044 to tell us more about your application – and how we can guide you through telecentric lensing and lighting options.

1st Vision’s sales engineers have over 100 years of combined experience to assist in your camera and components selection.  With a large portfolio of cameraslensescablesNIC cards and industrial computers, we can provide a full vision solution!

About you: We want to hear from you!  We’ve built our brand on our know-how and like to educate the marketplace on imaging technology topics…  What would you like to hear about?… Drop a line to info@1stvision.com with what topics you’d like to know more about.

How to calculate line rate on a line scan camera based on conveyor speed

Unless one calculates and sets the line rate correctly, there’s a risk of blur and sub-optimal performance. And/or purchasing a line scan camera that’s not up to the task; or that’s overkill and costs you more than would have been needed.

Line Scan concept – Courtesy Teledyne DALSA

Optional line scan review or introduction

Skip to the next section if you know line scan concepts already. Otherwise…

Perhaps you know about area scan imaging, where a 2D image is generated with a global shutter, exposing all pixels on a 2D sensor concurrently. And you’d like to understand line scan imaging by way of comparing it to area scan. See our blog What is the difference between an Area Scan and a Line Scan Camera?

30 minute informative overview of Line Scan imaging – Courtesy Teledyne DALSA

Maybe you prefer seeing a specific high-end product overview and application suggestions, such as the Teledyne DALSA 16k TDI line scan camera with 1MHz line rate. Or a view to tens of different line scan models, varying not only by manufacturer, but by sensor size and resolution, interface, and whether monochrome or color.

Either you recall how to determine resolution requirements in terms of pixel size relative to defect size, or you’ve chased the link in this sentence for a tutorial. So we’ll keep this blog as simple as possible, dealing with line rate calculation only.

Line scan cameras – Courtesy Teledyne DALSA

Calculate the line rate

Getting the line rate right is the application of the Goldilocks principle to line scanning.

Line rate too slow…Line rate too fast…
Blurred image if due to too long exposure, and/or missed segments due to skipped “slices”Oversampling can create confusion by identifying the same feature as two distinct features
Why we need to get the line rate rate right

A rotary encoder is typically used to synchronize the motion of the conveyor or web with the line scan camera (and lighting if pulsed). Naturally the system cannot be operated faster than the maximum line speed, but it may sometimes operator more slowly. This may happen during ramp up or slow down phases – when one may still need to obtain imaging – or by operator choice to conserve energy or avoid stressing mechanical systems.

Naming the variables … with example values

Resolution A = object space correlation to sensor; FOV / pixel array; e.g. if 550mm FOV and 2k sensor = 550/2000 = 0.275 pixels per mm

Transport speed T = mm per sec; e.g. 4k / 1mm yields rate of motion

Sampling frequency F = T / A; for example values above F = 4000 / 0.275 = 14545.4545 = 14.5kHz; spelled out: Frequency = Transport_speed / Pixel_spatial_resolution (what 1 pixel equals in target space)

For the example figures used above, a line scan camera with 2k resolution and a line scan frequency of about 14.5 kHz will be sufficient.

Download spreadsheet with labeled fields and examples:

Just click here, or on the image below, to download the spreadsheet calculator. It includes clearly labeled fields, and examples, as the companion piece for this blog:

Not included here… but happy to show you how

We’ve kept this blog intentionally lean, to avoid information overload. Additional values may also be calculated, of course, such as:

Data rate in MB / sec: Useful to confirm camera interface can sustain the data rate

Frame time: The amount of time to process each scanned image. Important to be sure the PC and image processing software are up to the task – based on empirical experience or by conferring with software provider.

1st Vision’s sales engineers have over 100 years of combined experience to assist in your camera and components selection.  With a large portfolio of cameraslensescablesNIC cards and industrial computers, we can provide a full vision solution!

About you: We want to hear from you!  We’ve built our brand on our know-how and like to educate the marketplace on imaging technology topics…  What would you like to hear about?… Drop a line to info@1stvision.com with what topics you’d like to know more about