Skip to content

Commit 455c125

Browse files
IRISZZWIRISZZW
authored
Merge pull request #45 from hustmzp/master
HUST Sound Snake
2 parents e6a1941 + cc06204 commit 455c125

24 files changed

+2387
-0
lines changed

‎arc_design_contest/2019/HUST_SoundSnake/.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Auto detect text files and perform LF normalization
2+
* text=auto

‎arc_design_contest/2019/HUST_SoundSnake/README.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Voice-operated snake game
2+
This application is an implement of off-line spoken keyword spotting using MLI Lib based on ARC IoT Development Kit.The optimized convolutional neural network can figure out ten basic control commands,even though the processing unit is not that powerful.Then we can use voice to control many things,for example,a snake game.
3+
DEMO VIDEO:[Video presentation](https://v.youku.com/v_show/id_XNDI5MDI2NjE2NA==.html?spm=a2h3j.8428770.3416059.1)
4+
5+
* [Introduction](#introduction)
6+
* [Function](#function)
7+
* [System Architecture](#system-architecture)
8+
* [Hardware and Software Setup](#hardware-and-software-setup)
9+
* [Required Hardware](#required-hardware)
10+
* [Required Software](#required-software)
11+
* [Hardware Connection](#hardware-connection)
12+
* [User Manual](#user-manual)
13+
* [Before Running This Application](#before-running-this-application)
14+
* [Run This Application](#run-this-application)
15+
16+
## Introduction
17+
18+
### Function
19+
20+
- Spotting the ten spoken commands:yes/no/up/down/left/right/on/off/stop/go.And the word will be printed through uart.
21+
- Play the game!Saying up/down/left/right commands,you can change the direction of the snake to catch the food or avoid crash with itself.
22+
23+
### System Architecture
24+
25+
![system_architecture][0]
26+
27+
## Hardware and Software Setup
28+
### Required Hardware
29+
- ARC IoT Development Kit
30+
31+
![ARC IoT Development Kit][1]
32+
33+
- WM8978 CODEC module
34+
35+
![WM8978 CODEC module][2]
36+
37+
- LED Matrix(MAX7219)
38+
39+
![LED Matrix][3]
40+
41+
### Required Software
42+
- ARC GNU Toolset 2018.09
43+
- emcARC OSP 2018.12 with MLI Lib
44+
- Serial port terminal, such as putty, tera-term or minicom
45+
46+
### Hardware Connection
47+
1.MAX7219 pins.
48+
49+
|MAX7219 pins|IoT DK pins |
50+
|------------|-------------|
51+
|CS |GPIO_8b0_PIN4|
52+
|DATA |GPIO_8b0_PIN5|
53+
|CLK |GPIO_8b1_PIN4|
54+
55+
2.Connect I2C pins of WM8978 to I2C0 of IoT DK.
56+
57+
3.Connect I2S pins of WM8978 to I2S_RX of IoT DK.**The REF_CLK should not be connected untile the initialization of WM8978 is finished**.
58+
59+
## User Manual
60+
61+
### Before Running This Application
62+
- Download source code of SoundSnake from github.
63+
- Make sure all the connections are correct.
64+
- The link script file has been modified to ensure the heap and stack are large enough and the addresses of program sections have also been redirected.Make sure the link script is correct.
65+
66+
### Run This Application
67+
68+
- Open serial terminal and configure it to right COM port and 115200bps.
69+
- Download with USB-JTAG or use bootloader to boot the program.
70+
- Connect the MCLK(REF_CLK) wire when seeing "Please connet the mclk !"at serial terminal.
71+
- Speak to the Mic.You'll see the word at the serial terminal and the snake will follow your command to change direction.
72+
73+
#### Makefile
74+
75+
- Selected MLI Lib here, then you can use MLI Lib API in your application:
76+
77+
LIB_SEL = embarc_mli
78+
79+
- Target options about IoT DK and toolchain:
80+
81+
BOARD = iotdk
82+
BD_VER = 10
83+
TOOLCHAIN = gnu
84+
85+
- The middleware used in your application:
86+
87+
MID_SEL = common
88+
89+
See [ embARC Example User Guide][40], **"Options to Hard-Code in the Application Makefile"** for more detailed information about **Makefile Options**.
90+
91+
#### Flie structure
92+
93+
| file | Function |
94+
| ------------------- | -----------------------------------------|
95+
| main.c | Initialization and main loop |
96+
| kws.c/kws.h | Keyword spotting(CNN) |
97+
| ds_cnn_weights.h | Weights list of CNN |
98+
| mfcc.c/mfcc.h | Algorithm of extracting mfcc features |
99+
| codec.c/codec.h | Recive and store audio frames(interrupt)| | snake.c/snake.h | Snake game and LED Matrix driver |
100+
| FFT.c/FFT.h | Fast Fourier transform |
101+
| wm8978.c/wm8978.h | Config wm8978 through I2C |
102+
| wm8978i2s.c/.h | I2S driver |
103+
104+
[0]: ./images/System_Architecture.PNG "system_architecture"
105+
[1]: ./images/ARC_IoT_Development_Kit.jpg "ARC IoT Development Kit"
106+
[2]: ./images/WM8978_CODEC_module.jpg "WM8978 CODEC module"
107+
[3]: ./images/LED_Matrix(MAX7219).jpg "LED Matrix"
108+
109+
[40]: http://embarc.org/embarc_osp/doc/embARC_Document/html/page_example.html " embARC Example User Guide"

‎arc_design_contest/2019/HUST_SoundSnake/images/ARC_IoT_Development_Kit.jpg‎

61.9 KB
Loading

‎arc_design_contest/2019/HUST_SoundSnake/images/LED_Matrix(MAX7219).jpg‎

69.7 KB
Loading

‎arc_design_contest/2019/HUST_SoundSnake/images/System_Architecture.PNG‎

84.2 KB
Loading

‎arc_design_contest/2019/HUST_SoundSnake/images/WM8978_CODEC_module.jpg‎

39.6 KB
Loading

‎arc_design_contest/2019/HUST_SoundSnake/src/FFT.c‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
#include "stdlib.h"
2+
#include "math.h"
3+
4+
//#define __EXTERN_W__
5+
6+
typedef struct
7+
{
8+
float real;
9+
float imag;
10+
} Complex;
11+
12+
#ifdef __EXTERN_W__
13+
//========Paste the definitions for W here========
14+
const Complex FFT_W[15] =
15+
{
16+
{0.98078528040323,-0.195090322016128},
17+
{0.923879532511287,-0.38268343236509},
18+
{0.831469612302545,-0.555570233019602},
19+
{0.707106781186547,-0.707106781186547},
20+
{0.555570233019602,-0.831469612302545},
21+
{0.38268343236509,-0.923879532511287},
22+
{0.195090322016128,-0.98078528040323},
23+
{5.55111512312578E-17,-1},
24+
{-0.195090322016128,-0.98078528040323},
25+
{-0.38268343236509,-0.923879532511287},
26+
{-0.555570233019602,-0.831469612302545},
27+
{-0.707106781186547,-0.707106781186547},
28+
{-0.831469612302545,-0.555570233019602},
29+
{-0.923879532511286,-0.38268343236509},
30+
{-0.98078528040323,-0.195090322016128}
31+
};
32+
const Complex *w = FFT_W;
33+
//================================================
34+
#endif
35+
36+
unsigned char fftx(Complex *d,int m)
37+
{
38+
#ifndef __EXTERN_W__
39+
static Complex *w;
40+
static int mw = 0;
41+
float arg, w_real, w_imag, wr_real, wr_imag, wtemp;
42+
#endif
43+
static int n = 1;
44+
Complex temp, tmp1, tmp2;
45+
Complex *di, *dip, *dj, *wp;
46+
int i, j, k, l, le, wi;
47+
#ifndef __EXTERN_W__
48+
if(m != mw)
49+
{
50+
if(mw != 0)
51+
free(w);
52+
mw = m;
53+
if(m == 0)
54+
return 0;
55+
n = 1 << m;
56+
le = n >> 1;
57+
w = (Complex*)calloc(le - 1,sizeof(Complex));
58+
if(!w)
59+
return 0;
60+
arg = 4.0 * atan(1.0) / le;
61+
wr_real = w_real = cos(arg);
62+
wr_imag = w_imag = -sin(arg);
63+
dj = w;
64+
for(j = 1; j < le; j++)
65+
{
66+
dj->real = (float)wr_real;
67+
dj->imag = (float)wr_imag;
68+
dj++;
69+
wtemp = wr_real * w_real - wr_imag * w_imag;
70+
wr_imag = wr_real * w_imag + wr_imag * w_real;
71+
wr_real = wtemp;
72+
}
73+
}
74+
#else
75+
n = 1 << m;
76+
#endif
77+
le = n;
78+
wi = 1;
79+
for(l = 0; l < m; l++)
80+
{
81+
le >>= 1;
82+
for(i = 0; i < n; i += (le << 1))
83+
{
84+
di = d + i;
85+
dip = di + le;
86+
temp.real = (di->real + dip->real);
87+
temp.imag = (di->imag + dip->imag);
88+
dip->real = (di->real - dip->real);
89+
dip->imag = (di->imag - dip->imag);
90+
*di = temp;
91+
}
92+
wp = (Complex*)w + wi - 1;
93+
for(j = 1; j < le; j++)
94+
{
95+
tmp1 = *wp;
96+
for(i = j; i < n; i += (le << 1))
97+
{
98+
di = d + i;
99+
dip = di + le;
100+
temp.real = (di->real + dip->real);
101+
temp.imag = (di->imag + dip->imag);
102+
tmp2.real = (di->real - dip->real);
103+
tmp2.imag = (di->imag - dip->imag);
104+
dip->real = (tmp2.real * tmp1.real - tmp2.imag * tmp1.imag);
105+
dip->imag = (tmp2.real * tmp1.imag + tmp2.imag * tmp1.real);
106+
*di = temp;
107+
}
108+
wp += wi;
109+
}
110+
wi <<= 1;
111+
}
112+
113+
for(i = 0; i < n; i++)
114+
{
115+
j = 0;
116+
for(k = 0; k < m; k++)
117+
j = (j << 1) | ((i >> k) & 1);
118+
if(i < j)
119+
{
120+
di = d + i;
121+
dj = d + j;
122+
temp = *dj;
123+
*dj = *di;
124+
*di = temp;
125+
}
126+
}
127+
return 1;
128+
}
129+
130+
void AmpSpectrum(Complex *pData,int m,float* pDCAmp,float* pDistortion)
131+
{
132+
// Amp = Abs / (1.414213562 * N)
133+
int i;
134+
int N = 1 << (m - 1);
135+
int N2 = 1 << m;
136+
float BasePower; //exactly BasePower*2*N*N;
137+
float SyntPower = 0; //exactly SyntPower*2*N*N;
138+
float ModelSquare;
139+
*pDCAmp = sqrt(pData[0].real * pData[0].real + pData[0].imag * pData[0].imag) / (N << 1);
140+
BasePower = pData[1].real * pData[1].real + pData[1].imag * pData[1].imag;
141+
pData[N2 - 1].real = sqrt(BasePower);
142+
pData[N2 - 1].imag = asin(pData[N2 - 1].imag / pData[N2 - 1].real);
143+
pData[N2 - 1].real /= N;
144+
for(i = 2; i < N; i++)
145+
{
146+
ModelSquare = pData[i].real * pData[i].real + pData[i].imag * pData[i].imag;
147+
pData[N2 - i].real = sqrt(ModelSquare);
148+
pData[N2 - i].imag = asin(pData[N2-i].imag/pData[N2-i].real);
149+
pData[N2 - i].real /= N;
150+
SyntPower += ModelSquare;
151+
}
152+
*pDistortion = sqrt(SyntPower / BasePower);
153+
}

‎arc_design_contest/2019/HUST_SoundSnake/src/FFT.h‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#include "stdlib.h"
2+
#include "math.h"
3+
4+
typedef struct
5+
{
6+
float real;
7+
float imag;
8+
} Complex;
9+
10+
unsigned char fftx(Complex *x,int m);

‎arc_design_contest/2019/HUST_SoundSnake/src/appl_mem_config.h‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#ifndef _APPL_MEM_CONFIG_H_
2+
#define _APPL_MEM_CONFIG_H_
3+
4+
#if defined(BOARD_EMSK)
5+
/* arcem9d only */
6+
7+
#define REGION_MLI_ROM REGION_ROM
8+
#define REGION_MLI_DATA REGION_DCCM
9+
#define REGION_MLI_BSS REGION_DCCM
10+
#define REGION_MLI_ZDATA REGION_DCCM
11+
#define REGION_MLI_MODEL_P2 REGION_DCCM
12+
#define REGION_MLI_MODEL REGION_DCCM
13+
#define REGION_MLI_XDATA REGION_XCCM
14+
#define REGION_MLI_YDATA REGION_YCCM
15+
16+
#elif defined(BOARD_IOTDK)
17+
18+
#define REGION_MLI_ROM REGION_ROM
19+
#define REGION_MLI_DATA REGION_DCCM
20+
#define REGION_MLI_BSS REGION_DCCM
21+
#define REGION_MLI_ZDATA REGION_DCCM
22+
#define REGION_MLI_MODEL_P2 REGION_DCCM
23+
#define REGION_MLI_MODEL REGION_XCCM
24+
#define REGION_MLI_XDATA REGION_XCCM
25+
#define REGION_MLI_YDATA REGION_YCCM
26+
27+
#elif defined(BOARD_EMSDP)
28+
29+
#define REGION_MLI_ROM REGION_ROM
30+
#define REGION_MLI_DATA REGION_DCCM
31+
#define REGION_MLI_BSS REGION_DCCM
32+
#define REGION_MLI_ZDATA REGION_DCCM
33+
#define REGION_MLI_MODEL_P2 REGION_DCCM
34+
#define REGION_MLI_MODEL REGION_XCCM
35+
#define REGION_MLI_XDATA REGION_XCCM
36+
#define REGION_MLI_YDATA REGION_YCCM
37+
38+
#elif defined(BOARD_NSIM)
39+
#define REGION_MLI_ROM REGION_ICCM
40+
#define REGION_MLI_DATA REGION_DCCM
41+
#define REGION_MLI_BSS REGION_DCCM
42+
#define REGION_MLI_ZDATA REGION_DCCM
43+
#define REGION_MLI_MODEL_P2 REGION_DCCM
44+
#define REGION_MLI_MODEL REGION_XCCM
45+
#define REGION_MLI_XDATA REGION_XCCM
46+
#define REGION_MLI_YDATA REGION_YCCM
47+
#endif
48+
49+
#endif /* _APPL_MEM_CONFIG_H_ */

0 commit comments

Comments
 (0)