Recently, Alibaba Dharma announced the launch of its self-developed ISP image recognition processor, which can make self-driving cars “see” more clearly. Through the self-developed 3D noise reduction and image enhancement algorithm, the image recognition ability of the vehicle front-end camera can be improved by more than 10%. At present, the ISP processor has been used in self-driving logistics vehicles.

What exactly is an ISP?

As we all know, autonomous driving technology has attracted the attention of technology giants in the technology industry, and one of the most important aspects is to ensure driving safety. It is often said that planning ahead so that autonomous vehicles can “see” the external conditions so that they can respond to various emergencies earlier is the current focus of ensuring the safety of autonomous driving.

In an autonomous vehicle, the front-end camera is like the eye, which can collect various image data, and the ISP technology is equivalent to the vision of the car, which can improve the characteristics and accuracy of the collected image scene.

ISP is the abbreviation of English Image Signal Processor. The full name is image processor. It is mainly used to process the output signal of the front-end image sensor to match the image sensors of different manufacturers. Generally speaking, the main functions of ISP are related to images, including multiple image algorithm processing modules, including linearization, denoising, white balance, sharpening, dead pixel removal, dark current deduction, color enhancement, etc.

At present, the place where people can access the most ISP is in the aspect of mobile phone imaging. At present, most mobile phones on the market integrate ISP chips, and some products use independent ISP chips, which are mainly used in the camera function of mobile phones. The signal output by the image sensor is post-processed, and the ISP chip can also realize functions such as face recognition and automatic scene recognition.

Alibaba’s self-developed ISP processor allows vehicles to “see” more clearly

Industry experts pointed out that due to low cost and mature technology, the vehicle front-end camera has become the most widely used sensor in autonomous driving, and the powerful ISP processor can improve the intelligence level of the autonomous driving industry chain.

The main function of ISP on the vehicle is to perform arithmetic processing on the signal output by the front-end image sensor CMOS. As an important component of the vehicle camera, it can help the vehicle to clearly “see” the external environment image.

Alibaba’s self-developed ISP processor is inseparable from the unique 3D noise reduction and image enhancement algorithm support of DAMO Academy, which can greatly improve the image recognition capability of vehicle cameras during the day and night.

The road test results of the Autopilot Lab of DAMO Academy show that using the DAMO Academy ISP processor, the vehicle-mounted camera can detect and recognize image objects by more than 10% compared to the mainstream processors in the industry even in the night scene where the field of view is originally unclear. In the above, the originally ambiguous labels can also be clearly identified, so that the vehicle can achieve more accurate judgments and greatly improve the safety of automatic driving.

In the research direction of the autonomous driving laboratory of Alibaba Dharma Academy, the first item is the introduction of environmental perception. It mentioned that it is necessary to study multi-sensor collaborative perception and fusion technology to realize the key information collection and knowledge extraction of the surrounding environment by autonomous vehicles, including drivable areas, lane lines, traffic signs, traffic lights, vehicles, pedestrians and all obstacles Detection and recognition of objects, prediction of the pose and motion state of obstacles such as vehicles and pedestrians; through a reasonable multi-sensor layout configuration scheme to ensure 360-degree coverage around the vehicle and enhanced coverage of key areas; through the safety redundancy design software The system solution ensures the accurate perception of the surrounding environment information of the vehicle, thereby ensuring the safe and stable operation of the autonomous vehicle.

Therefore, Alibaba’s self-developed ISP image recognition processor enables autonomous vehicles to “see” more clearly, which is of great significance to the vehicle’s accurate image judgment, which is not only conducive to improving vehicle safety, but also conducive to the vehicle’s next decision-making.

Ali’s “China Core” Layout

Of course, on the track of self-developed ISP processors, the strength of Ali’s opponents should not be underestimated. There are mobile product giants such as Qualcomm, MediaTek, Huawei Hisilicon, Samsung, and Apple, which are mainly concentrated in the SoC field. Giants in other fields such as Ambarella, TI, NXP, Shengmai, etc. Overall, the competition is relatively fierce.

Many people first realized that Ali is an e-commerce giant from Tmall, Taobao, and Alipay. In recent years, Ali’s transformation also tells people that it has been making progress on the road to “China Core”.

From the acquisition of Zhongtianwei to the establishment of Damo Academy, the establishment of Pingtouge semiconductor Company, to the successive investments in 5 chip companies including Cambrian, Barefoot Networks, Shenjian, Kneron, and Aojie Technology, Alibaba’s strategic layout in chips also gradually surfaced.

In the past year, Alibaba has successively launched the first RISC-V processor Xuantie 910, the SoC chip platform “Wujian”, and the first AI chip “Hanguang 800”, which have attracted widespread attention in China. The launch of several major products also announced Ali’s determination to make cores incessantly.

(The picture comes from OFweek Weike.com)

As Zhang Jianfeng, chief technology officer of Alibaba Group, said: “In the global chip field, Alibaba is a newcomer, and Xuantie and Hanguang 800 are the first steps of Pingtou’s long march. Today, Alibaba has enough capabilities to do traditional hardware. What companies can do, including what they can’t do. What they can do is hardware, what they can’t do is that we turn software into hardware. Because our software has algorithms. The combination of the two enables us to use With the capability of hardware and the speed of Internet companies, it took only one and a half years to complete the design, verification and tapeout of the chip. We have full confidence that Alibaba will become a true collaborative development of hardware and software integration in the future. technology company”.