nba
Chinese team develops intelligent vision sensor that turns light signals into AI-ready tokens, reducing use of energy_我的网站
一 |
A Chinese research team from Nanjing University has developed a new ultra-low-power intelligent vision sensor chip, dubbed "LightTok," that can convert light signals into tokens within the sensor, significantly reducing the high energy consumption caused by frequent transfers of massive amounts of redundant data, the principal investigator told the Global Times.
According to a release from the Institute of Brain-Inspired Intelligence of Nanjing University, tokens generated by the LightTok chip can be directly fed into a Transformer encoder for image recognition.
"Our design idea was to move token generation onto the sensor itself, allowing the chip to directly produce tokens that AI models can process once light reaches the sensor," Miao Feng, director of the Institute of Brain-Inspired Intelligence at Nanjing University, told the Global Times on Thursday. "These tokens contain complete image information."
Physical AI refers to intelligent systems capable of autonomously perceiving, reasoning, acting and receiving feedback in the real world, representing a key pathway for AI to move from the digital realm into the physical world. Vision-based physical AI systems powered by large AI models need to convert visual information from real-world environments into tokens that can be processed by AI models before feeding these tokens into Transformers for subsequent tasks.
In traditional visual perception pipelines, light signals must go through multiple stages, including image sensing, analog-to-digital conversion, data buffering and transfer, digital image patching and embedding, before being transformed into tokens that AI models can process. The frequent transfer of massive amounts of redundant data has resulted in high energy consumption at the edge, according to a report by Science and Technology Daily.
The LightTok chip directly addresses a key challenge in physical AI hardware: how to efficiently acquire and tokenize visual information from the physical world with low energy consumption, Miao said.
The LightTok chip consists of a photosensitive memory array and peripheral circuits. The team built the array based on single-layer molybdenum disulfide (MoS₂) floating-gate phototransistors, with each pixel capable of sensing light, storing information and performing analog computing. By processing optical information directly within the chip, the device can convert captured visual signals into tokens for AI models, according to the research team.
The current LightTok prototype has a resolution of 32×32 pixels, or 1,024 photosensitive pixels, which is still smaller than that of smartphone cameras and industrial imaging systems. However, Miao said the technology is compatible with CMOS manufacturing processes and can be scaled up. With wafer-level growth of molybdenum disulfide materials, the chip could potentially achieve a scale comparable to existing imaging devices.
Miao said the chip also draws inspiration from the information-processing mechanism of human vision. Similar to how the retina extracts key visual information before transmitting it to the brain, the chip also aims to process visual information at an early stage.
The research was conducted in collaboration with another research team from the National University of Singapore. The findings were published on Wednesday in Nature Sensors, an internationally renowned journal in the field of sensing technology, according to the release.
Potential applications for LightTok include drone systems, autonomous remote sensing and small-scale embodied AI systems, Miao said. In these scenarios, devices need to continuously detect, understand and track targets, generating massive amounts of visual data. By reducing the energy required for visual processing, the technology could extend the operating time of drones, satellites and small robots with limited power supplies, the expert said.
。
二 | 水利部消息，据预报，受冷暖空气共同影响，8月25日至28日，西北中部东部、华北中部北部将有一次较强降雨过程，以中到大雨为主，其中内蒙古中南部、陕西北部、山西北部、河北中部、天津等地部分地区将有暴雨到大暴雨；黄河上中游干流及支流湟水、大黑河、皇甫川、窟野河，海河流域永定河、大清河、滹沱河等河流将出现涨水过程，暴雨区内部分中小河流可能发生超警以上洪水，山丘区山洪灾害风险较高。8月24日，水利部组织滚动会商，分析研判全国雨情水情汛情形势，对西北、华北地区暴雨洪水防御工作作出针对性安排部署。指导督促相关地方水利部门和流域管理机构锚定“人员不伤亡、水库不垮坝、重要堤防不决口、重要基础设施不受冲击”目标，落实落细暴雨洪水防御各项措施。
三 | 一是强化监测预报预警。密切监视天气形势和雨水情发展变化，滚动预报预警，预警信息直达一线。二是确保水库安全度汛。严格落实水库安全度汛责任，特别是强化中小水库、病险水库巡查防守，逐库落实防漫坝垮坝措施。
四 | 三是强化淤地坝安全度汛。
五 | 督促淤地坝防汛“三个责任人”全部上岗到位，加强巡查值守和险情处置，发现险情征兆第一时间转移受威胁人员。四是做好中小河流洪水防御。切实落实中小河流堤防巡查防守责任，预置抢险力量、料物、设备，做到险情早发现、早处置、早消除，必要时提请地方政府提前转移危险区群众。五是突出抓好山洪灾害防御。落实预警“叫应”机制，抓好“谁组织、转移谁、何时转、转何处、不擅返”五个关键环节责任和措施，确保人民群众生命安全。
Current article:http://www.suiyangguiduanzudoushaopao.buzz/list_5oew/bswgma.htm
Published on:15:56:39