Cognitive Computational Vision-Based (CVM) Monitoring • Real-Time Autonomous Species Identification. • 100% IFQ Accountable web- based trip-by-trip vessel EM Near Real Time data acquisition. • Autonomous computer vision length determination for stock assessment scientists. CVM • Improve the cost-effectiveness and capacity of our Fisheries Information Systems (FIS) for providing adequate field-based observations. • Accelerated implementation of electronic monitoring methods (e.g., digital imaging and electronic log books) to complement fishery observer and catch monitoring programs. • Advancement of cognitive computing proof-of- concepts in other Agency natural resource monitoring efforts. Shorebased CVM unit installation, Bornstein Seafood, Astoria, Oregon Expected Results, Relative Importance Colby Brady 1 , Guy *Paillet 2, , Anne Menendez 2 , Andrew Bornstein 3 , Kevin Dunn 4 , & Sheryl Flores 5 Consistent baseline CVM data augmented and calibrated by port sampler sampling data. Real Time web-based species ID ? . Human EM video review methodologies have limited species determination capabilities. However, in fish cognitive computing platforms offer promise to insure that species determination is consistently well clarified. We have incorporated novel parallel processing silicon system-on- chip video sensor cognitive computing hardware linked to ruggedized military Central Processing Unit (CPU) hardware to investigate the functionality of a cognitive computational real-time and near real-time (NRT) IFQ web-based data acquisition, and pattern recognition strategy. The video analytic capability of the unique hardware and customized software platform is also being examined to determine if vessel deck discard behavior training and recognition is possible as a workable proof-of-concept strategic IFQ EM strategy. Specifically, these shoreside and vessel CVM prototypes (or “Cognitive DVRs”) consists of “Cognisight” Miniature Trainable Vision Sensors (MTVS) linked to a miniaturized military computer, further linked to a “CogniBlox” configurable NeuroMem vision board, enabling “smart” on-site real-time massively parallel data mining and sensor fusion. Sampling site: Bornstein Seafood, Astoria, Oregon Cognitive Computing hardware platform tools may be uniquely suited for pattern recognition and ecosystem intelligence applications. Introduction Oregon Department of Fish and Wildlife port sampler support for verified species-specific video annotation (Image Knowledge Builder) West Coast IFQ fishing vessel Installation, Captain Kevin Dunn F/V Iron Lady, Warrenton, Oregon 1 National Oceanic and Atmospheric Administration (NOAA-WCR), 2 General Vision Inc., 3 Bornstein Seafood (Astoria, OR), 4 F/V Iron Lady, 5 Oregon Department of Fish and Wildlife (ODFW, Astoria) COMPLETED: • Hardware engineering design and manufacture of Windows-based CVM equipment delivery. • Discard detection software development complete, available for testing. Previous test was through a sensor aimed at a screen. Recent update now able to scan hard drive EM data. • Web-based streaming and Cognitive “neuron” object training. • Wireless remote desktop control with Panasonic Toughpad for captain, port sampler, and observer testing. • “Fish” recognition conducted (species-specific training pending). • Autonomous frame grabs of individual fish from 24 live video feed (i.e., initial test and “fish” training captured >3,000 individual fish images on the conveyor belt within two hours during offload). TO BE COMPLETED: • Custom Species-specific image library, Zero Instruction Set Computing (ZISC) real-time algorithm development (which have a completely different software engineering architecture from traditional post- analysis CPU algorithms). • Autonomous length determination algorithm development from individual fish frame grabs. TO BE COMPLETED: • Installation of CVM unit tentatively scheduled for the end of January. • Test manual and autonomous EM full trip data acquisition of all sorting events. • Compare between windows-based ATOM processor CVM prototype unit and linux/ android-based ARM processor CVM unit. • Discard detection capability (post- observer verification, from shoreside species Image Knowledge Builder (IKB) image library. • Trip-by-trip data web-based data acquisition. Modern CPU processors (far left): • Memory bottleneck. • High power consumption. CM1K pattern recognition chip (above and left): • Memory and processing logic combined in the same element. • Parallel architecture of identical elements. • Simple access to all elements connected in parallel. 512 GB SSD (600 Mbytes/second) SATA 2 Inside Top Inside Bottom Dual Core i.MX6XX ARM Running Android WiFi Module For Tablet connection Gigabit Ethernet 2 USB HS HDMI Output (test) Power supply 24 Volts input Outside IP68 Interface up to 8 cameras HD_SDI 1080p synchronized CogniBlox (2) Real time Cognitive Video NOAA Fisheries staff, in collaboration with ODFW biological port sampler staff, have developed dock-side species-specific CVM training protocols that will enable ongoing Image Knowledge Builder (IKB) library development and feature extraction annotation for ongoing ZISC algorithm development. Valuable ODFW port sampling staff collaboration will enable acquisition of verified data-poor species-specific data, including length and sex information on a fish-by-fish basis. ODFW port sampler staff will be able to access web-based PacFin databases immediately in the field using a Panasonic toughpad (windows –based), thereby reducing potential transcription errors and reconciliation validation. If successful, CVM will create additional baseline information streams, which would enable limited port sampling energies to be more targeted and focused on the tasks that humans are particularly adept at gathering (DNA, scale samples, sex determination), while allowing CVM to gather real-time digital data that cognitive computing systems are more adept at gathering (100% computer vision census of lengths, consistent data-poor complex speciation, etc.). The National Marine Fisheries Service (NMFS), West Coast Region (WCR) is exploring Computational Vision-Based Monitoring (CVM) in the Individual Fishing Quota (IFQ) groundfish trawl fishery. The goal of the research is see if CVM can reduce monitoring costs while providing better and more timely data as compared to current (1) electronic monitoring/ reporting (EM, ER) hardware; (2) speciation methodologies, and; (3) logistical issues with periodic collecting and replacing of hard drives in the field. This project is done in collaboration with General Vision Inc. (a cognitive computing hardware and software developer), Bornstein Seafoods, F/V Iron Lady, and ODFW. Funding for this research project was via a Fisheries Information System (FIS) grant. This project aims to determine if CVM can be used to automatically determine species and length of individual fish at a shoreside processor offloading conveyor belt. This project also aims to determine if an automatic field-based web CVM data transmission strategy could be developed to reduce the need for contractors to retrieve EM data in the field by manually pulling removable hard drives. If successful, a trip-by-trip automatic data transmission strategy could dramatically reduce data verification times and costs to fishermen and management. The CVM unit has initially been deployed on a bottom trawl processing plant conveyor- sorting belt. In addition to use in a processing plant, a further step will be to place a CVM prototype unit on a IFQ bottom trawl fishing vessel.