Skip to content

Commit a5a1cda

Browse files
authored
Merge pull request #57 from KaiWenyy/master
ntu magical robot
2 parents 21076f5 + fc7ff69 commit a5a1cda

25 files changed

+5513
-0
lines changed
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.vscode/
2+
*.log
3+
obj_*/
4+
5+
*.swp
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# iRobot - Front-Vehicle Detection to Enhance Drivers' Awareness
2+
3+
This application, which implements a image semantic segmentation on ARC IoT board, is designed to show how to use OV7670 camera and Machine Learning in embARC to achieve front-vehicle detection to enhance drivers' awareness. However, ultimately, we want to demonstrate that ARC processor can achieve scene understanding to do much more insteresting application in the future. For example, smart movement robot like household robot cleaner.
4+
5+
* [Introduction](#introduction)
6+
* [iRobot System Architecture](#system-architecture)
7+
* [Hardware and Software Setup](#hardware-and-software-setup)
8+
* [Required Hardware](#required-hardware)
9+
* [Required Software](#required-software)
10+
* [Hardware Connection](#hardware-connection)
11+
* [User Manual](#user-manual)
12+
* [Before Running This Application](#before-running-this-application)
13+
* [Run This Application](#run-this-application)
14+
15+
## iRobot Introduction
16+
iRobot is trying to enhance dirver's awarenessby by utilizing image segmentation. We can segment the vehicles in front of the current driver and warn if necessary.
17+
18+
19+
### System Architecture
20+
![][3]
21+
22+
## Hardware and Software Setup
23+
### Required Hardware
24+
- [ARC IoT Development Kit][1]
25+
- [OV7670 Camera Module][2]
26+
- [Arduino]
27+
- [HC-05]
28+
- [Raspberry Pi]
29+
- [Micro-Robot Car]
30+
31+
### Required Software
32+
- ARC GNU Toolset 2019
33+
- Serial port terminal, such as putty, tera-term or minicom
34+
- embARC MLI
35+
36+
### Hardware Connection
37+
1. Connect OV7670 camera module to ARC following below instructions
38+
39+
# ARC: 2x18 Pin Extension Header
40+
SIOC -> I2C0_SCL (need pull-up 10K resistor)
41+
SIOD -> I2C0_SDA (need pull-up 10K resistor)
42+
43+
# ARC: Arduino PIN
44+
VSYBC -> arduino IO0
45+
PCLK -> arduino IO1
46+
XCLK -> arduino IO3
47+
D7~D0 -> arduino IO4~IO11
48+
49+
3V3 -> +3.3V
50+
RESET -> +3.3V
51+
GND -> GND
52+
PWDN -> GND
53+
54+
2. (Optional) Connect HC-05 to arduino and prepared the necessary micro-robot car
55+
56+
3. Boot up ARC IoT Development Kit Board to start ML segmentation
57+
58+
4. (Optional) Boot up Raspberry Pi and connect it to ARC IoT Development Kit Board to transmitted results to the laptop via WiFi
59+
60+
5. Connect the USB port of ARC to the laptop to recieve image segmentation results
61+
62+
## User Manual
63+
### Before Running This Application
64+
* Download source code of iRobot from github
65+
* (Optional) Prepare a Raspberry pi and download ***rpi_src*** directory in iRobot
66+
* Make sure all connection is correct again
67+
* (Optional) Make sure iRobot is in the WiFi environment, including WiFi name, password in rpi_src
68+
* Please goto $ARC_ROOT/middleware/common/console_io.c and find xprintf_setup.
69+
* Change `console_uart->uart_open(1500000)` to make sure the baudrate is 1,000,000
70+
71+
### Run This Application
72+
* Type `make run` in the terminal to download program into ARC IoTdk board
73+
* (Optional) After Rpi connect the wifi, the laptop should received the segmentation results in 0.5 fps
74+
* After ARC connect with the laptop via USB, type `sudo python3 python/cameraGrabber.py` in the terminal to start recieving the segmentation results in 0.5 fps
75+
* To test with the simple demo, type `sudo python3 python/car_count.py` or `sudo python3 python/danger_detect.py` in the terminal to start the applications
76+
77+
### Demo Video Link
78+
* https://v.youku.com/v_show/id_XNDM0OTI5Mzk5Mg==.html?spm=a2h0j.11185381.listitem_page1.5~A
79+
80+
[1]: https://embarc.org/embarc_osp/doc/build/html/board/iotdk.html "ARC IoT Development Kit"
81+
[2]: https://www.voti.nl/docs/OV7670.pdf "OV7670 Camera Module"
82+
[3]: ./doc/system.png "System Architecture"
1000 KB
Loading
1.79 MB
Loading
1.13 MB
Loading
1 MB
Loading
492 KB
Loading
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#ifndef _APPL_MEM_CONFIG_H_
2+
#define _APPL_MEM_CONFIG_H_
3+
4+
#if defined(BOARD_EMSK)
5+
/* arcem9d only */
6+
7+
#define REGION_MLI_ROM REGION_ROM
8+
#define REGION_MLI_DATA REGION_DCCM
9+
#define REGION_MLI_BSS REGION_DCCM
10+
#define REGION_MLI_ZDATA REGION_DCCM
11+
#define REGION_MLI_MODEL_P2 REGION_DCCM
12+
#define REGION_MLI_MODEL REGION_DCCM
13+
#define REGION_MLI_XDATA REGION_XCCM
14+
#define REGION_MLI_YDATA REGION_YCCM
15+
16+
#elif defined(BOARD_IOTDK)
17+
18+
#define REGION_MLI_ROM REGION_ROM
19+
#define REGION_MLI_DATA REGION_DCCM
20+
#define REGION_MLI_BSS REGION_DCCM
21+
#define REGION_MLI_ZDATA REGION_DCCM
22+
#define REGION_MLI_MODEL_P2 REGION_DCCM
23+
#define REGION_MLI_MODEL REGION_XCCM
24+
#define REGION_MLI_XDATA REGION_XCCM
25+
#define REGION_MLI_YDATA REGION_YCCM
26+
27+
#elif defined(BOARD_EMSDP)
28+
29+
#define REGION_MLI_ROM REGION_ROM
30+
#define REGION_MLI_DATA REGION_DCCM
31+
#define REGION_MLI_BSS REGION_DCCM
32+
#define REGION_MLI_ZDATA REGION_DCCM
33+
#define REGION_MLI_MODEL_P2 REGION_DCCM
34+
#define REGION_MLI_MODEL REGION_XCCM
35+
#define REGION_MLI_XDATA REGION_XCCM
36+
#define REGION_MLI_YDATA REGION_YCCM
37+
38+
#elif defined(BOARD_NSIM)
39+
#define REGION_MLI_ROM REGION_ICCM
40+
#define REGION_MLI_DATA REGION_DCCM
41+
#define REGION_MLI_BSS REGION_DCCM
42+
#define REGION_MLI_ZDATA REGION_DCCM
43+
#define REGION_MLI_MODEL_P2 REGION_DCCM
44+
#define REGION_MLI_MODEL REGION_XCCM
45+
#define REGION_MLI_XDATA REGION_XCCM
46+
#define REGION_MLI_YDATA REGION_YCCM
47+
#endif
48+
49+
#endif /* _APPL_MEM_CONFIG_H_ */
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
#pragma once
2+
#include <stdint.h>
3+
4+
enum RESOLUTION{VGA,QVGA,QQVGA};
5+
enum COLORSPACE{YUV422,RGB565,BAYER_RGB};
6+
struct regval_list{
7+
uint8_t reg_num;
8+
uint8_t value;
9+
};
10+
void setColorSpace(enum COLORSPACE color);
11+
void setRes(enum RESOLUTION res);
12+
void camInit(void);
13+
#define OV7670_I2C_ADDRESS 0x21
14+
/* Registers */
15+
#define REG_GAIN 0x00 /* Gain lower 8 bits (rest in vref) */
16+
#define REG_BLUE 0x01 /* blue gain */
17+
#define REG_RED 0x02 /* red gain */
18+
#define REG_VREF 0x03 /* Pieces of GAIN, VSTART, VSTOP */
19+
#define REG_COM1 0x04 /* Control 1 */
20+
#define COM1_CCIR656 0x40 /* CCIR656 enable */
21+
#define REG_BAVE 0x05 /* U/B Average level */
22+
#define REG_GbAVE 0x06 /* Y/Gb Average level */
23+
#define REG_AECHH 0x07 /* AEC MS 5 bits */
24+
#define REG_RAVE 0x08 /* V/R Average level */
25+
#define REG_COM2 0x09 /* Control 2 */
26+
#define COM2_SSLEEP 0x10 /* Soft sleep mode */
27+
#define REG_PID 0x0a /* Product ID MSB */
28+
#define REG_VER 0x0b /* Product ID LSB */
29+
#define REG_COM3 0x0c /* Control 3 */
30+
#define COM3_SWAP 0x40 /* Byte swap */
31+
#define COM3_SCALEEN 0x08 /* Enable scaling */
32+
#define COM3_DCWEN 0x04 /* Enable downsamp/crop/window */
33+
#define REG_COM4 0x0d /* Control 4 */
34+
#define REG_COM5 0x0e /* All "reserved" */
35+
#define REG_COM6 0x0f /* Control 6 */
36+
#define REG_AECH 0x10 /* More bits of AEC value */
37+
#define REG_CLKRC 0x11 /* Clocl control */
38+
#define CLK_EXT 0x40 /* Use external clock directly */
39+
#define CLK_SCALE 0x3f /* Mask for internal clock scale */
40+
#define REG_COM7 0x12 /* Control 7 */
41+
#define COM7_RESET 0x80 /* Register reset */
42+
#define COM7_FMT_MASK 0x38
43+
#define COM7_FMT_VGA 0x00
44+
#define COM7_FMT_CIF 0x20 /* CIF format */
45+
#define COM7_FMT_QVGA 0x10 /* QVGA format */
46+
#define COM7_FMT_QCIF 0x08 /* QCIF format */
47+
#define COM7_RGB 0x04 /* bits 0 and 2 - RGB format */
48+
#define COM7_YUV 0x00 /* YUV */
49+
#define COM7_BAYER 0x01 /* Bayer format */
50+
#define COM7_PBAYER 0x05 /* "Processed bayer" */
51+
#define REG_COM8 0x13 /* Control 8 */
52+
#define COM8_FASTAEC 0x80 /* Enable fast AGC/AEC */
53+
#define COM8_AECSTEP 0x40 /* Unlimited AEC step size */
54+
#define COM8_BFILT 0x20 /* Band filter enable */
55+
#define COM8_AGC 0x04 /* Auto gain enable */
56+
#define COM8_AWB 0x02 /* White balance enable */
57+
#define COM8_AEC 0x01 /* Auto exposure enable */
58+
#define REG_COM9 0x14 /* Control 9- gain ceiling */
59+
#define REG_COM10 0x15 /* Control 10 */
60+
#define COM10_HSYNC 0x40 /* HSYNC instead of HREF */
61+
#define COM10_PCLK_HB 0x20 /* Suppress PCLK on horiz blank */
62+
#define COM10_HREF_REV 0x08 /* Reverse HREF */
63+
#define COM10_VS_LEAD 0x04 /* VSYNC on clock leading edge */
64+
#define COM10_VS_NEG 0x02 /* VSYNC negative */
65+
#define COM10_HS_NEG 0x01 /* HSYNC negative */
66+
#define REG_HSTART 0x17 /* Horiz start high bits */
67+
#define REG_HSTOP 0x18 /* Horiz stop high bits */
68+
#define REG_VSTART 0x19 /* Vert start high bits */
69+
#define REG_VSTOP 0x1a /* Vert stop high bits */
70+
#define REG_PSHFT 0x1b /* Pixel delay after HREF */
71+
#define REG_MIDH 0x1c /* Manuf. ID high */
72+
#define REG_MIDL 0x1d /* Manuf. ID low */
73+
#define REG_MVFP 0x1e /* Mirror / vflip */
74+
#define MVFP_MIRROR 0x20 /* Mirror image */
75+
#define MVFP_FLIP 0x10 /* Vertical flip */
76+
77+
#define REG_AEW 0x24 /* AGC upper limit */
78+
#define REG_AEB 0x25 /* AGC lower limit */
79+
#define REG_VPT 0x26 /* AGC/AEC fast mode op region */
80+
#define REG_HSYST 0x30 /* HSYNC rising edge delay */
81+
#define REG_HSYEN 0x31 /* HSYNC falling edge delay */
82+
#define REG_HREF 0x32 /* HREF pieces */
83+
#define REG_TSLB 0x3a /* lots of stuff */
84+
#define TSLB_YLAST 0x04 /* UYVY or VYUY - see com13 */
85+
#define REG_COM11 0x3b /* Control 11 */
86+
#define COM11_NIGHT 0x80 /* NIght mode enable */
87+
#define COM11_NMFR 0x60 /* Two bit NM frame rate */
88+
#define COM11_HZAUTO 0x10 /* Auto detect 50/60 Hz */
89+
#define COM11_50HZ 0x08 /* Manual 50Hz select */
90+
#define COM11_EXP 0x02
91+
#define REG_COM12 0x3c /* Control 12 */
92+
#define COM12_HREF 0x80 /* HREF always */
93+
#define REG_COM13 0x3d /* Control 13 */
94+
#define COM13_GAMMA 0x80 /* Gamma enable */
95+
#define COM13_UVSAT 0x40 /* UV saturation auto adjustment */
96+
#define COM13_UVSWAP 0x01 /* V before U - w/TSLB */
97+
#define REG_COM14 0x3e /* Control 14 */
98+
#define COM14_DCWEN 0x10 /* DCW/PCLK-scale enable */
99+
#define REG_EDGE 0x3f /* Edge enhancement factor */
100+
#define REG_COM15 0x40 /* Control 15 */
101+
#define COM15_R10F0 0x00 /* Data range 10 to F0 */
102+
#define COM15_R01FE 0x80 /* 01 to FE */
103+
#define COM15_R00FF 0xc0 /* 00 to FF */
104+
#define COM15_RGB565 0x10 /* RGB565 output */
105+
#define COM15_RGB555 0x30 /* RGB555 output */
106+
#define REG_COM16 0x41 /* Control 16 */
107+
#define COM16_AWBGAIN 0x08 /* AWB gain enable */
108+
#define REG_COM17 0x42 /* Control 17 */
109+
#define COM17_AECWIN 0xc0 /* AEC window - must match COM4 */
110+
#define COM17_CBAR 0x08 /* DSP Color bar */
111+
/*
112+
* This matrix defines how the colors are generated, must be
113+
* tweaked to adjust hue and saturation.
114+
*
115+
* Order: v-red, v-green, v-blue, u-red, u-green, u-blue
116+
* They are nine-bit signed quantities, with the sign bit
117+
* stored in0x58.Sign for v-red is bit 0, and up from there.
118+
*/
119+
#define REG_CMATRIX_BASE0x4f
120+
#define CMATRIX_LEN 6
121+
#define REG_CMATRIX_SIGN0x58
122+
#define REG_BRIGHT 0x55 /* Brightness */
123+
#define REG_CONTRAS 0x56 /* Contrast control */
124+
#define REG_GFIX 0x69 /* Fix gain control */
125+
#define REG_REG76 0x76 /* OV's name */
126+
#define R76_BLKPCOR 0x80 /* Black pixel correction enable */
127+
#define R76_WHTPCOR 0x40 /* White pixel correction enable */
128+
#define REG_RGB444 0x8c /* RGB 444 control */
129+
#define R444_ENABLE 0x02 /* Turn on RGB444, overrides 5x5 */
130+
#define R444_RGBX 0x01 /* Empty nibble at end */
131+
#define REG_HAECC1 0x9f /* Hist AEC/AGC control 1 */
132+
#define REG_HAECC2 0xa0 /* Hist AEC/AGC control 2 */
133+
#define REG_BD50MAX 0xa5 /* 50hz banding step limit */
134+
#define REG_HAECC3 0xa6 /* Hist AEC/AGC control 3 */
135+
#define REG_HAECC4 0xa7 /* Hist AEC/AGC control 4 */
136+
#define REG_HAECC5 0xa8 /* Hist AEC/AGC control 5 */
137+
#define REG_HAECC6 0xa9 /* Hist AEC/AGC control 6 */
138+
#define REG_HAECC7 0xaa /* Hist AEC/AGC control 7 */
139+
#define REG_BD60MAX 0xab /* 60hz banding step limit */
140+
#define COM7_FMT_CIF 0x20 /* CIF format */
141+
#define COM7_RGB 0x04 /* bits 0 and 2 - RGB format */
142+
#define COM7_YUV 0x00 /* YUV */
143+
#define COM7_BAYER 0x01 /* Bayer format */
144+
#define COM7_PBAYER 0x05 /* "Processed bayer" */
145+
#define COM10_VS_LEAD 0x04 /* VSYNC on clock leading edge */
146+
#define COM11_50HZ 0x08 /* Manual 50Hz select */
147+
#define COM13_UVSAT 0x40 /* UV saturation auto adjustment */
148+
#define COM15_R01FE 0x80 /* 01 to FE */
149+
#define MTX1 0x4f /* Matrix Coefficient 1 */
150+
#define MTX2 0x50 /* Matrix Coefficient 2 */
151+
#define MTX3 0x51 /* Matrix Coefficient 3 */
152+
#define MTX4 0x52 /* Matrix Coefficient 4 */
153+
#define MTX5 0x53 /* Matrix Coefficient 5 */
154+
#define MTX6 0x54 /* Matrix Coefficient 6 */
155+
#define MTXS 0x58 /* Matrix Coefficient Sign */
156+
#define AWBC7 0x59 /* AWB Control 7 */
157+
#define AWBC8 0x5a /* AWB Control 8 */
158+
#define AWBC9 0x5b /* AWB Control 9 */
159+
#define AWBC10 0x5c /* AWB Control 10 */
160+
#define AWBC11 0x5d /* AWB Control 11 */
161+
#define AWBC12 0x5e /* AWB Control 12 */
162+
#define REG_GFI 0x69 /* Fix gain control */
163+
#define GGAIN 0x6a /* G Channel AWB Gain */
164+
#define DBLV 0x6b
165+
#define AWBCTR3 0x6c /* AWB Control 3 */
166+
#define AWBCTR2 0x6d /* AWB Control 2 */
167+
#define AWBCTR1 0x6e /* AWB Control 1 */
168+
#define AWBCTR0 0x6f /* AWB Control 0 */
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#ifndef SCCB_H
2+
#define SCCB_H
3+
#include "embARC.h"
4+
#include "embARC_debug.h"
5+
#include "embARC_error.h"
6+
7+
#define OV7670_SCCB_ADDR 0x21
8+
#define OV7670_CHECK_EXP_NORTN(EXPR) CHECK_EXP_NOERCD(EXPR, error_exit)
9+
10+
static DEV_IIC_PTR ov7670_sccb = NULL;
11+
12+
int sccbInit(uint8_t devId);
13+
14+
bool writeReg(uint8_t regAddr, uint8_t data);
15+
16+
uint8_t readReg(uint8_t regAddr);
17+
18+
#endif

0 commit comments

Comments
 (0)