Skip to content

Commit 703291a

Browse files
author
James Cori
committed
Merge branch 'develop'
2 parents f94c842 + cd8de6a commit 703291a

File tree

64 files changed

+5710
-2483
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+5710
-2483
lines changed

‎.circleci/config.yml‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,29 @@ jobs:
5959
APPNAME: "resources-api"
6060
steps: *builddeploy_steps
6161

62+
"Run-Newman-Test":
63+
docker:
64+
- image: circleci/node:12
65+
- image: tray/dynamodb-local
66+
command: "-inMemory -port 7777"
67+
- image: elasticsearch:6.8.13
68+
environment:
69+
discovery.type: "single-node"
70+
steps:
71+
- checkout
72+
- setup_remote_docker
73+
- run:
74+
name: 'newman test'
75+
command: |
76+
npm i
77+
cd local
78+
cd ..
79+
npm run create-tables:test
80+
cd mock
81+
(npm run mock-challenge-api&)
82+
(NODE_ENV=test npm start&)
83+
npm run test:newman
84+
6285
workflows:
6386
version: 2
6487
build:
@@ -77,4 +100,15 @@ workflows:
77100
context : org-global
78101
filters:
79102
branches:
80-
only: master
103+
only: master
104+
105+
- "Run-Newman-Test":
106+
context : org-global
107+
requires:
108+
- build-dev
109+
- build-prod
110+
filters:
111+
branches:
112+
only:
113+
- automated-postman-testing
114+
- develop

‎.eslintrc.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
extends: standard

‎PoC-test.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Postman PoC test
2+
## Prerequisite
3+
- start db and es:
4+
```bash
5+
$ cd resources-api/local
6+
$ docker-compose up
7+
```
8+
- create tables:
9+
```bash
10+
$ cd resources-api
11+
12+
# NOTE:
13+
# if tables and data already exist, please run first
14+
15+
# $ npm run drop-tables:test
16+
17+
# to drop data and tables
18+
19+
$ npm run create-tables:test
20+
```
21+
- start mock challenge api:
22+
```
23+
$ cd resources-api/mock
24+
$ npm run mock-challenge-api
25+
```
26+
- start app
27+
```bash
28+
$ cd resources-api
29+
$ source env.sh # set env variables
30+
$ NODE_ENV=test npm start
31+
```
32+
33+
## newman test
34+
```bash
35+
$ npm run test:newman
36+
```
37+
38+
## Postman test
39+
Please refer to: https://drive.google.com/file/d/1VcTtNwI5_TXgnEKT4TruqG0Z-ts0J83G/view?usp=sharing
40+
41+
42+
## Postman mock server
43+
E2E tests use nock to mock `BUSAPI_URL`, where postman mock server could be used to replace nock.
44+
Please refer to: https://drive.google.com/file/d/1GXMzyqpzwix-LDBwieiRFfpJlJxrTIgI/view?usp=sharing

‎ReadMe.md‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ This microservice provides interaction with Challenge Resources.
2525
## Prerequisites
2626
- [NodeJS](https://nodejs.org/en/) (v10)
2727
- [DynamoDB](https://aws.amazon.com/dynamodb/)
28+
- [ElasticSearch](https://www.elastic.co/)
2829
- [Docker](https://www.docker.com/)
2930
- [Docker Compose](https://docs.docker.com/compose/)
3031

@@ -58,6 +59,11 @@ The following parameters can be set in config files or in env variables:
5859
- DYNAMODB.AWS_WRITE_UNITS: The DynamoDB table write unit configuration, default is 2
5960
- DYNAMODB.TIMEOUT: The timeout setting used in health check
6061
- SCOPES: The M2M scopes, refer `config/default.js` for more information
62+
- ES.HOST: Elasticsearch host, default value is 'localhost:9200'
63+
- ES.API_VERSION: Elasticsearch API version, default value is '6.8'
64+
- ES.ES_INDEX: Elasticsearch index name for resources, default value is 'resources'
65+
- ES.ES_TYPE: Elasticsearch index type for resources, default value is '_doc'
66+
- ES.ES_REFRESH: Elasticsearch force refresh flag, default value is 'true'
6167
- BUSAPI_URL: the bus api, default value is 'https://api.topcoder-dev.com/v5'
6268
- KAFKA_ERROR_TOPIC: Kafka error topic, default value is 'common.error.reporting',
6369
- KAFKA_MESSAGE_ORIGINATOR: the Kafka message originator, default value is 'resources-api'
@@ -75,7 +81,11 @@ Configuration for testing is at `config/test.js`, only add such new configuratio
7581
- Run lint `npm run lint`
7682
- Run lint fix `npm run lint:fix`
7783
- Create tables `npm run create-tables`
84+
- Drop tables `npm run drop-tables`
85+
- Create tables for test environment `npm run create-tables:test`
86+
- Drop tables for test environment `npm run drop-tables:test`
7887
- Clear and init db `npm run init-db`
88+
- Initialize ElasticSearch `npm run init-es`
7989
- Start app `npm start`
8090
- App is running at `http://localhost:3000`
8191
- Start mock server `npm run mock-challenge-api`
@@ -87,7 +97,7 @@ To install foreman follow this [link](https://theforeman.org/manuals/1.24/#3.Ins
8797

8898
To know how to use foreman follow this [link](https://theforeman.org/manuals/1.24/#2.Quickstart)
8999

90-
100+
91101
### DynamoDB Setup
92102

93103
We can use DynamoDB setup on Docker for testing purpose. Just run `docker-compose up` in `local` folder.
@@ -100,6 +110,18 @@ You can also use your own AWS DynamoDB service for testing purpose.
100110
2. Make sure you have configured all config parameters. Refer [Configuration](#configuration)
101111
3. Run `npm run create-tables` to create tables.
102112

113+
### ElasticSearch Setup
114+
115+
We can use ElasticSearch on Docker for testing purpose. Just run `docker-compose up` in `local` folder.
116+
117+
You can also use your own remote ElasticSearch service for testing purpose.
118+
119+
### Create ElasticSearch Index
120+
121+
1. Makre sure ElasticSearch are running as per instructions above.
122+
2. Make sure you have configured all config parameters. Refer [Configuration](#configuration)
123+
3. Run `npm run init-es force` to create index.
124+
103125
### Mock Challenge V5 API
104126

105127
The `GET /v5/challenges/{id}` is mocked. It is a simple server app, the code is under mock folder.
@@ -132,6 +154,8 @@ The following test parameters can be set in config file or in env variables:
132154

133155
- Start Local DynamoDB.
134156
- Create DynamoDB tables.
157+
- Start Local ElasticSearch.
158+
- Create ElasticSearch index.
135159
- Various config parameters should be properly set.
136160

137161
### Running unit tests
@@ -160,3 +184,7 @@ npm run e2e
160184
## Verification
161185

162186
Refer to the verification document `Verification.md`
187+
188+
## Postman PoC test
189+
190+
Refer to the PoC test document [**PoC-test.md**](PoC-test.md)

‎Verification.md‎

Lines changed: 60 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -10,69 +10,68 @@
1010

1111
## Unit test Coverage
1212

13-
115 passing (2m)
14-
15-
----------------------------------------|----------|----------|----------|----------|-------------------|
16-
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
17-
----------------------------------------|----------|----------|----------|----------|-------------------|
18-
All files | 88.21 | 75.6 | 90.14 | 88.66 | |
19-
resources-api | 100 | 100 | 100 | 100 | |
20-
app-bootstrap.js | 100 | 100 | 100 | 100 | |
21-
app-constants.js | 100 | 100 | 100 | 100 | |
22-
resources-api/config | 100 | 92.65 | 100 | 100 | |
23-
default.js | 100 | 92.65 | 100 | 100 | 13,14,16,17,29 |
24-
test.js | 100 | 100 | 100 | 100 | |
25-
resources-api/src/common | 77.78 | 54.41 | 84.09 | 79.04 | |
26-
errors.js | 100 | 50 | 100 | 100 | 23 |
27-
helper.js | 65.63 | 52.17 | 75.86 | 67.39 |... 91,209,262,272 |
28-
logger.js | 92.31 | 60 | 100 | 92.31 | 31,53,58,82,116 |
29-
resources-api/src/models | 100 | 50 | 100 | 100 | |
30-
Resource.js | 100 | 100 | 100 | 100 | |
31-
ResourceRole.js | 100 | 100 | 100 | 100 | |
32-
ResourceRolePhaseDependency.js | 100 | 100 | 100 | 100 | |
33-
index.js | 100 | 50 | 100 | 100 | 14 |
34-
resources-api/src/services | 95.1 | 78.57 | 100 | 95 | |
35-
ResourceRolePhaseDependencyService.js | 93.1 | 79.17 | 100 | 92.98 | 55,77,113,136 |
36-
ResourceRoleService.js | 94.87 | 75 | 100 | 94.59 | 47,82 |
37-
ResourceService.js | 96.26 | 78.75 | 100 | 96.23 | 77,154,158,162 |
38-
----------------------------------------|----------|----------|----------|----------|-------------------|
13+
127 passing (1m)
3914

15+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
16+
----------------------------------------|----------|----------|----------|----------|-------------------
17+
All files | 88.69 | 83.7 | 90.43 | 88.48 |
18+
resources-api | 100 | 100 | 100 | 100 |
19+
app-bootstrap.js | 100 | 100 | 100 | 100 |
20+
app-constants.js | 100 | 100 | 100 | 100 |
21+
resources-api/config | 100 | 98.78 | 100 | 100 |
22+
default.js | 100 | 98.78 | 100 | 100 | 35
23+
test.js | 100 | 100 | 100 | 100 |
24+
resources-api/src/common | 74.45 | 52.17 | 83.02 | 75.11 |
25+
errors.js | 100 | 50 | 100 | 100 | 23
26+
helper.js | 65.13 | 50 | 76.32 | 65.75 |... 46,364,374,390
27+
logger.js | 92.31 | 60 | 100 | 92.31 | 31,53,58,82,116
28+
resources-api/src/models | 100 | 50 | 100 | 100 |
29+
MemberProfile.js | 100 | 100 | 100 | 100 |
30+
MemberStats.js | 100 | 100 | 100 | 100 |
31+
Resource.js | 100 | 100 | 100 | 100 |
32+
ResourceRole.js | 100 | 100 | 100 | 100 |
33+
ResourceRolePhaseDependency.js | 100 | 100 | 100 | 100 |
34+
index.js | 100 | 50 | 100 | 100 | 8,18
35+
resources-api/src/services | 98.23 | 96.45 | 100 | 98.05 |
36+
ResourceRolePhaseDependencyService.js | 94.83 | 87.5 | 100 | 94.74 | 77,113,136
37+
ResourceRoleService.js | 96.55 | 90 | 100 | 95.35 | 60,96
38+
ResourceService.js | 100 | 100 | 100 | 100 |
4039

4140
## E2E test Coverage
4241

43-
164 passing (6m)
42+
170 passing (1m)
4443

45-
-------------------------------------------|----------|----------|----------|----------|-------------------|
46-
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
47-
-------------------------------------------|----------|----------|----------|----------|-------------------|
48-
All files | 93.92 | 80.27 | 99 | 93.81 | |
49-
resources-api | 95.88 | 82.98 | 100 | 95.79 | |
50-
app-bootstrap.js | 100 | 100 | 100 | 100 | |
51-
app-constants.js | 100 | 100 | 100 | 100 | |
52-
app-routes.js | 97.3 | 93.75 | 100 | 97.3 | 24 |
53-
app.js | 94.23 | 77.42 | 100 | 94.23 | 42,72,88 |
54-
resources-api/config | 100 | 92.65 | 100 | 100 | |
55-
default.js | 100 | 92.65 | 100 | 100 | 13,14,16,17,29 |
56-
test.js | 100 | 100 | 100 | 100 | |
57-
resources-api/src | 100 | 100 | 100 | 100 | |
58-
routes.js | 100 | 100 | 100 | 100 | |
59-
resources-api/src/common | 89.47 | 70.59 | 97.73 | 89.22 | |
60-
errors.js | 100 | 50 | 100 | 100 | 23 |
61-
helper.js | 86.46 | 73.91 | 96.55 | 85.87 |... 91,209,262,272 |
62-
logger.js | 92.31 | 65 | 100 | 92.31 | 31,53,58,82,116 |
63-
resources-api/src/controllers | 100 | 100 | 100 | 100 | |
64-
HealthCheckController.js | 100 | 100 | 100 | 100 | |
65-
ResourceController.js | 100 | 100 | 100 | 100 | |
66-
ResourceRoleController.js | 100 | 100 | 100 | 100 | |
67-
ResourceRolePhaseDependencyController.js | 100 | 100 | 100 | 100 | |
68-
resources-api/src/models | 100 | 50 | 100 | 100 | |
69-
Resource.js | 100 | 100 | 100 | 100 | |
70-
ResourceRole.js | 100 | 100 | 100 | 100 | |
71-
ResourceRolePhaseDependency.js | 100 | 100 | 100 | 100 | |
72-
index.js | 100 | 50 | 100 | 100 | 14 |
73-
resources-api/src/services | 94.93 | 78.07 | 100 | 94.84 | |
74-
HealthCheckService.js | 92.31 | 50 | 100 | 92.31 | 15 |
75-
ResourceRolePhaseDependencyService.js | 93.1 | 79.17 | 100 | 92.98 | 55,77,113,136 |
76-
ResourceRoleService.js | 94.87 | 75 | 100 | 94.59 | 47,82 |
77-
ResourceService.js | 96.26 | 78.75 | 100 | 96.23 | 77,154,158,162 |
78-
-------------------------------------------|----------|----------|----------|----------|-------------------|
44+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
45+
-------------------------------------------|----------|----------|----------|----------|-------------------
46+
All files | 95.8 | 89.63 | 99.17 | 95.57 |
47+
resources-api | 96.26 | 85.96 | 100 | 96.12 |
48+
app-bootstrap.js | 100 | 100 | 100 | 100 |
49+
app-constants.js | 100 | 100 | 100 | 100 |
50+
app-routes.js | 97.62 | 96.15 | 100 | 97.62 | 25
51+
app.js | 94.23 | 77.42 | 100 | 94.23 | 45,75,91
52+
resources-api/config | 100 | 98.78 | 100 | 100 |
53+
default.js | 100 | 98.78 | 100 | 100 | 35
54+
test.js | 100 | 100 | 100 | 100 |
55+
resources-api/src | 100 | 100 | 100 | 100 |
56+
routes.js | 100 | 100 | 100 | 100 |
57+
resources-api/src/common | 91.19 | 75 | 98.11 | 90.95 |
58+
errors.js | 100 | 50 | 100 | 100 | 23
59+
helper.js | 90.13 | 78.57 | 97.37 | 89.73 |... 69,364,374,390
60+
logger.js | 92.31 | 65 | 100 | 92.31 | 31,53,58,82,116
61+
resources-api/src/controllers | 100 | 100 | 100 | 100 |
62+
HealthCheckController.js | 100 | 100 | 100 | 100 |
63+
ResourceController.js | 100 | 100 | 100 | 100 |
64+
ResourceRoleController.js | 100 | 100 | 100 | 100 |
65+
ResourceRolePhaseDependencyController.js | 100 | 100 | 100 | 100 |
66+
resources-api/src/models | 100 | 50 | 100 | 100 |
67+
MemberProfile.js | 100 | 100 | 100 | 100 |
68+
MemberStats.js | 100 | 100 | 100 | 100 |
69+
Resource.js | 100 | 100 | 100 | 100 |
70+
ResourceRole.js | 100 | 100 | 100 | 100 |
71+
ResourceRolePhaseDependency.js | 100 | 100 | 100 | 100 |
72+
index.js | 100 | 50 | 100 | 100 | 8,18
73+
resources-api/src/services | 98.24 | 96.45 | 100 | 98.06 |
74+
HealthCheckService.js | 100 | 100 | 100 | 100 |
75+
ResourceRolePhaseDependencyService.js | 94.83 | 87.5 | 100 | 94.74 | 77,113,136
76+
ResourceRoleService.js | 96.55 | 90 | 100 | 95.35 | 60,96
77+
ResourceService.js | 100 | 100 | 100 | 100 |

‎app-bootstrap.js‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,10 @@
33
*/
44

55
global.Promise = require('bluebird')
6+
const config = require('config')
67
const Joi = require('joi')
78

89
Joi.optionalId = () => Joi.string().uuid()
910
Joi.id = () => Joi.optionalId().required()
1011
Joi.page = () => Joi.number().integer().min(1).default(1)
11-
Joi.perPage = () => Joi.number().integer().min(1).max(100).default(20)
12+
Joi.perPage = () => Joi.number().integer().min(1).max(10000).default(config.DEFAULT_PAGE_SIZE)

‎app-constants.js‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,11 @@ const UserRoles = {
99
User: 'Topcoder User'
1010
}
1111

12+
const ChallengeStatuses = {
13+
Completed: 'Completed'
14+
}
15+
1216
module.exports = {
13-
UserRoles
17+
UserRoles,
18+
ChallengeStatuses
1419
}

‎config/default.js‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ module.exports = {
66
LOG_LEVEL: process.env.LOG_LEVEL || 'debug',
77
PORT: process.env.PORT || 3000,
88
API_VERSION: process.env.API_VERSION || 'v5',
9-
DEFAULT_PAGE_SIZE: process.env.DEFAULT_PAGE_SIZE || 1000,
9+
DEFAULT_PAGE_SIZE: process.env.DEFAULT_PAGE_SIZE || 50,
1010
// used to properly set the header response to api calls for services behind a load balancer
1111
API_BASE_URL: process.env.API_BASE_URL || `http://localhost:3000`,
1212

@@ -29,11 +29,12 @@ module.exports = {
2929
CHALLENGE_PHASES_API_URL: process.env.CHALLENGE_PHASES_API_URL || 'https://api.topcoder-dev.com/v5/challenge-phases',
3030

3131
DYNAMODB: {
32-
// AWS_ACCESS_KEY_ID: process.env.AWS_FAKE_ID,
33-
// AWS_SECRET_ACCESS_KEY: process.env.AWS_FAKE_KEY,
32+
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID,
33+
AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY,
3434
AWS_REGION: process.env.AWS_REGION || 'us-east-1',
3535
IS_LOCAL_DB: process.env.IS_LOCAL_DB ? process.env.IS_LOCAL_DB === 'true' : true,
36-
URL: process.env.DYNAMODB_URL || 'http://localhost:8000',
36+
DYNAMODB_URL: process.env.DYNAMODB_URL || 'http://localhost:7777',
37+
URL: process.env.DYNAMODB_URL || 'http://localhost:7777',
3738
AWS_READ_UNITS: process.env.AWS_READ_UNITS || 4,
3839
AWS_WRITE_UNITS: process.env.AWS_WRITE_UNITS || 2,
3940
TIMEOUT: process.env.DYNAMODB_TIMEOUT || 10000

‎config/test.js‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,14 @@
33
*/
44

55
module.exports = {
6+
TERMS_API_URL: 'http://localhost:4000/v5/terms',
7+
BUSAPI_URL: 'https://65279208-4c97-4d1a-a925-041b2433787a.mock.pstmn.io/v5',
8+
CHALLENGE_PHASES_API_URL: 'http://localhost:4000/v5/challenge-phases',
9+
DYNAMODB: {
10+
AWS_ACCESS_KEY_ID: 'FAKE_ACCESS_KEY',
11+
AWS_SECRET_ACCESS_KEY: 'FAKE_SECRET_ACCESS_KEY',
12+
URL: 'http://localhost:7777'
13+
},
614
WAIT_TIME: 1500,
715
MOCK_CHALLENGE_API_PORT: 4000
816
}

0 commit comments

Comments
 (0)