Typical Camera ISP Pipeline
An image sensor captures raw light measurements at the pixels, not a finished image. The ISP converts that raw data into usable RGB or YUV images.
- DPC: Defective Pixel Correction
- BLC: Black Level Correction
- HDR: High Dynamic Range
- NR: Noise Reduction
- RGB: Red-Green-Blue
- YUV: Luminance-Chrominance
- CCM: Color Correction Matrix
- LSC: Lens Shading Correction
- WB: White Balance
- CFA: Color Filter Array
- AF: Auto Focus
- LDC: Lens Distortion Correction
- TNR: Temporal Noise Reduction
ISP Tuning: The ISP must be tuned for the specific sensor, lens, processing platform, and application. Proper tuning maintains consistent image quality across changing lighting and operating conditions for human viewing or AI processing.
On-Camera vs. On-Chip ISP
The ISP can be an on-camera ISP located in the camera, or an on-chip ISP integrated in the host SoC. A discrete ISP, such as the iND880, can also be used within the camera or on a separate processing board.
Sony ISX031 and ISX028
Cameras using Sony ISX031 and ISX028 sensors are examples of the on-camera ISP approach. They use a stacked pixel-and-logic architecture with Sony’s proprietary ISP integrated with the sensor. As a Sony partner, D3 Embedded uses Sony tuning tools and direct factory support to optimize image quality for customer requirements.
On-Camera ISP advantages
- Efficient Multi-Stream processing: On-camera ISPs simplify multi-stream processing and synchronization, preventing SoC overload. Line-based processing eliminates external DRAM requirements while enabling precise, real-time synchronization. This reduces SoC DRAM bandwidth and processing overhead for stable multi-channel performance.
- Very Low Latency: <1ms compared to at least 1 frame latency (33ms for 30fps) for on-chip ISP
- Faster initialization time since you don’t have to wait for the SoC to load registers.
- SoC Hardware-Agnostic: On-camera ISP decouples the imaging pipeline from SoC limitations, bugs and update cycles. This accelerates time-to-market by eliminating repeated ISP Tuning and simplifying hardware transitions. A standardized imaging environment ensures consistent output and preserves AI model integrity across platforms.
Balanced Performance Architecture: On-camera processing is optimized for efficiency and relies on effective passive heat dissipation through the camera housing to maintain image quality. The power envelope is specifically tuned for resolutions up to 8MP, balancing processing speed and thermal performance.
On-Chip ISP advantages
- Cost and Design Efficiency: Eliminates the need for a dedicated on-camera processor, reducing camera cost. However, it requires external DRAM, which can add latency and system cost.
- High-Resolution Scalability: Leverages the superior computational power of the host SoC to support resolutions beyond 8MP, enabling high-resolution and high-frame-rate applications.
- Advanced Image Processing: Some SoC’s provide sophisticated, hardware-accelerated features such as Motion-Compensated Temporal Noise Reduction (MCTNR) and AI-enhanced pipelines.
On-Chip ISP Image Tuning
On-chip ISP Tuning is typically more complex and time-consuming, requiring deep expertise. Our image science team uses proprietary tools and direct support from Texas Instruments, NVIDIA, Qualcomm, and Intel to provide expert tuning that meets specific customer requirements for image quality.
As an Elite member of the NVIDIA Partner Network, D3 Embedded has extensive experience with NVIDIA Jetson ISP tuning and SIPL (NvSIPL) camera integration.
Learn more about D3 Embedded’s Camera ISP Tuning and Image Quality Services.
