Skip to content

Commit 43d792c

Browse files
phead1987phead1987
committed
init commit
0 parents  commit 43d792c

37 files changed

+16879
-0
lines changed

‎ReadMe.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# Topcoder Member API v5
2+
3+
## Dependencies
4+
5+
- nodejs https://nodejs.org/en/ (v10)
6+
- prisma
7+
- PostgreSQL
8+
- Docker, Docker Compose
9+
10+
## Database Setup
11+
12+
Please setup PostgreSQL first. If you are using docker, you can run:
13+
```bash
14+
docker run -d --name memberdb -p 5432:5432 \
15+
-e POSTGRES_USER=johndoe -e POSTGRES_DB=memberdb \
16+
-e POSTGRES_PASSWORD=mypassword \
17+
postgres:16
18+
```
19+
20+
After that, please set db URL environment variables:
21+
```bash
22+
export DATABASE_URL="postgresql://johndoe:mypassword@localhost:5432/memberdb"
23+
```
24+
25+
This variable is important since it's required by prisma.
26+
27+
If you want to do anything with database, this variable is necessary.
28+
29+
## Database Scripts
30+
31+
Before running db scripts, please make sure you have setup db and config db url as above.
32+
33+
```bash
34+
# set db url values
35+
export DATABASE_URL="postgresql://johndoe:mypassword@localhost:5432/memberdb"
36+
37+
# install dependencies
38+
npm install
39+
```
40+
41+
Here are some helpful scripts for db operations.
42+
43+
```bash
44+
# create db tables
45+
npm run init-db
46+
47+
# Clear all db data
48+
npm run clear-db
49+
50+
# create test data
51+
npm run seed-data
52+
53+
# Reset db
54+
npm run reset-db
55+
```
56+
57+
## Database Seed Data
58+
59+
I have created a script to download data from dev environment and a script to load them into db.
60+
61+
To use them, you should:
62+
- Make sure you have started db.
63+
- Check configs in `src/scripts/config.js`. Add some handle if you like.
64+
- Open a terminal and navigate to codebase folder. Set `DATABASE_URL` above.
65+
- Run `npm install`.
66+
- Use `node src/scripts/download.js` to download profile data.
67+
- Run `npm run clear-db` to clear db data first
68+
- Run `npm run seed-data` to import data into db
69+
70+
71+
## Configuration
72+
73+
Configuration for the application is at `config/default.js`.
74+
The following parameters can be set in config files or in env variables:
75+
76+
- LOG_LEVEL: the log level, default is 'debug'
77+
- PORT: the server port, default is 3000
78+
- AUTH_SECRET: The authorization secret used during token verification.
79+
- VALID_ISSUERS: The valid issuer of tokens.
80+
- AUTH0_URL: AUTH0 URL, used to get M2M token
81+
- AUTH0_PROXY_SERVER_URL: AUTH0 proxy server URL, used to get M2M token
82+
- AUTH0_AUDIENCE: AUTH0 audience, used to get M2M token
83+
- TOKEN_CACHE_TIME: AUTH0 token cache time, used to get M2M token
84+
- AUTH0_CLIENT_ID: AUTH0 client id, used to get M2M token
85+
- AUTH0_CLIENT_SECRET: AUTH0 client secret, used to get M2M token
86+
- BUSAPI_URL: Bus API URL
87+
- KAFKA_ERROR_TOPIC: Kafka error topic used by bus API wrapper
88+
- GROUPS_API_URL: Groups API URL
89+
- AMAZON.AWS_ACCESS_KEY_ID: The Amazon certificate key to use when connecting.
90+
- AMAZON.AWS_SECRET_ACCESS_KEY: The Amazon certificate access key to use when connecting.
91+
- AMAZON.AWS.SESSION_TOKEN: The user session token, used when developing locally against the TC dev AWS services
92+
- AMAZON.AWS_REGION: The Amazon certificate region to use when connecting.
93+
- AMAZON.PHOTO_S3_BUCKET: the AWS S3 bucket to store photos
94+
- AMAZON.S3_API_VERSION: the AWS S3 API version
95+
- FILE_UPLOAD_SIZE_LIMIT: the file upload size limit in bytes
96+
- PHOTO_URL_TEMPLATE: photo URL template, its <key> will be replaced with S3 object key
97+
- VERIFY_TOKEN_EXPIRATION: verify token expiration in minutes
98+
- EMAIL_VERIFY_AGREE_URL: email verify agree URL, the <emailVerifyToken> will be replaced with generated verify token
99+
- EMAIL_VERIFY_DISAGREE_URL: email verify disagree URL
100+
- SCOPES: the configurable M2M token scopes, refer `config/default.js` for more details
101+
- MEMBER_SECURE_FIELDS: Member profile identifiable info fields, only admin, M2M, or member himself can fetch these fields
102+
- COMMUNICATION_SECURE_FIELDS: Member contact information, accessible by admins, managers, and copilots - anyone with a role in the AUTOCOMPLETE_ROLES array
103+
- MEMBER_TRAIT_SECURE_FIELDS: Member traits identifiable info fields, only admin, M2M, or member himself can fetch these fields
104+
- MISC_SECURE_FIELDS: Misc identifiable info fields, only admin, M2M, or member himself can fetch these fields
105+
- STATISTICS_SECURE_FIELDS: Member Statistics identifiable info fields, only admin, M2M, or member himself can fetch these fields
106+
- HEALTH_CHECK_TIMEOUT: health check timeout in milliseconds
107+
108+
Set the following environment variables used by bus API to get TC M2M token (use 'set' insted of 'export' for Windows OS):
109+
```
110+
export AUTH0_CLIENT_ID=
111+
export AUTH0_CLIENT_SECRET=
112+
export AUTH0_URL=
113+
export AUTH0_AUDIENCE=
114+
```
115+
116+
Also properly configure AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, PHOTO_S3_BUCKET config parameters.
117+
118+
Test configuration is at `config/test.js`. You don't need to change them.
119+
The following test parameters can be set in config file or in env variables:
120+
121+
- ADMIN_TOKEN: admin token
122+
- USER_TOKEN: user token
123+
- EXPIRED_TOKEN: expired token
124+
- INVALID_TOKEN: invalid token
125+
- M2M_FULL_ACCESS_TOKEN: M2M full access token
126+
- M2M_READ_ACCESS_TOKEN: M2M read access token
127+
- M2M_UPDATE_ACCESS_TOKEN: M2M update (including 'delete') access token
128+
- S3_ENDPOINT: endpoint of AWS S3 API, for unit and e2e test only; default to `localhost:9000`
129+
130+
## AWS S3 Setup
131+
Go to https://console.aws.amazon.com/ and login. Choose S3 from Service folder and click `Create bucket`. Following the instruction to create S3 bucket.
132+
After creating bucket, click `Permissions` tab of the bucket, in the `Block public access` section, disable the block, so that public access
133+
is allowed, then we can upload public accessible photo to the bucket.
134+
135+
## Local Mock Service
136+
137+
To make local development easier, I create a mock server at `mock`.
138+
139+
You can start it with `node mock/mock-api.js` and it will listen to port `4000`
140+
141+
## Local Configs
142+
143+
Please run following commands to set necessary configs:
144+
145+
```bash
146+
export AUTH0_URL="http://localhost:4000/v5/auth0"
147+
export BUSAPI_URL="http://localhost:4000/v5"
148+
export AUTH0_CLIENT_ID=xyz
149+
export AUTH0_CLIENT_SECRET=xyz
150+
export LOOKER_API_BASE_URL="http://localhost:4000/v5/looker"
151+
export LOOKER_API_CLIENT_ID=xyz
152+
export LOOKER_API_CLIENT_SECRET=xyz
153+
export USERFLOW_PRIVATE_KEY=mysecret
154+
```
155+
156+
These commands will set auth0, event bus pi and looker api to local mock server.
157+
158+
## Local Deployment
159+
160+
- Make sure you have started db and set `DATABASE_URL`.
161+
- Make sure you have create db structure. Seed data is optional.
162+
- Install dependencies `npm install`
163+
- Start app `npm start`
164+
- App is running at port 3000. You can visit `http://localhost:3000/v6/members/health`
165+
166+
167+
## Tests
168+
169+
170+
Make sure you have followed above steps to
171+
- setup db and config db url
172+
- setup local mock api and set local configs
173+
- it will really call service and mock api
174+
175+
Unit tests use `aws-sdk-mock` to mock S3 operations. So you can safely run tests without S3 configs.
176+
177+
Then you can run:
178+
```bash
179+
npm run test
180+
```
181+
182+
## Verification
183+
Refer to the verification document `Verification.md`

‎Verification.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
2+
## JWT
3+
4+
To make it easier to review, I create a script at `src/scripts/member-jwt.js` to generate JWT.
5+
6+
You need to make sure:
7+
- `secret` equals to config `AUTH_SECRET`
8+
- `iss` is in config `VALID_ISSUERS`
9+
- `exp` is a timestamp is after current time so this token is not expired
10+
11+
After that, you can change the payload by yourself.
12+
13+
For example, you want to udpate member profile as user "Ghostar", you need to
14+
- query Ghostar's user id in db
15+
- update jwt payload userId field
16+
- generate new jwt with this script
17+
18+
Sometimes the application will check `authUser.userId === member.userId`.
19+
20+
## Member Endpoints
21+
22+
You can setup seed data to test these features. Here are some useful tips.
23+
24+
### Update Member Email
25+
26+
In this endpoint, user may update email.
27+
28+
If user updates email, following things will happen:
29+
- member table `newEmail`, `emailVerifyToken`, `emailVerifyTokenDate` `newEmailVerifyToken`, `newEmailVerifyTokenDate` will be updated
30+
- tokens are new uuid and used in verifyEmail endpoint.
31+
- Use GET /members/<handle>/verify?token=<token> to verify token.
32+
- user need to verify both email and new email
33+
- after both emails are verified, user's email value will be updated to new email
34+
35+
### Member Photo Upload
36+
37+
There are a lot of S3 operations in this endpoint that are not updated.
38+
39+
To make it all working, it will take you much time and efforts.
40+
41+
To test this endpoint, you can comment them all and directly set photoUrl to db.
42+
43+
That is the only thing I update in this challenge.
44+
45+
## Member Trait Endpoints
46+
47+
All endpoints are simple CRUD. You should be able to check them by yourself.
48+
49+
Just be careful about the schemas used for different kind of trait.
50+
51+
52+
## Prisma Schema Updates
53+
54+
There are some changes to prisma schema.
55+
56+
- Add memberTraits.hobby
57+
- Remove displayMode.memberSkills @ignore
58+
- Add stats fields as discussed in forum
59+

‎app-bootstrap.js‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
/**
2+
* App bootstrap
3+
*/
4+
global.Promise = require('bluebird')
5+
const Joi = require('joi')
6+
7+
Joi.page = () => Joi.number().integer().min(1).default(1)
8+
Joi.perPage = () => Joi.number().integer().min(1).max(100).default(50)
9+
Joi.size = () => Joi.number().integer().min(1).max(1000).default(500)
10+
Joi.sort = () => Joi.string().default('asc')

‎app-constants.js‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
/**
2+
* App constants
3+
*/
4+
const ADMIN_ROLES = ['administrator', 'admin']
5+
const SEARCH_BY_EMAIL_ROLES = ADMIN_ROLES.concat('tgadmin');
6+
const AUTOCOMPLETE_ROLES = ['copilot', 'administrator', 'admin', 'Connect Copilot', 'Connect Account Manager', 'Connect Admin', 'Account Executive']
7+
8+
const EVENT_ORIGINATOR = 'topcoder-member-api'
9+
10+
const EVENT_MIME_TYPE = 'application/json'
11+
12+
const TOPICS = {
13+
MemberCreated: 'member.action.profile.create',
14+
MemberUpdated: 'member.action.profile.update',
15+
EmailChanged: 'member.action.email.profile.emailchange.verification',
16+
MemberTraitCreated: 'member.action.profile.trait.create',
17+
MemberTraitUpdated: 'member.action.profile.trait.update',
18+
MemberTraitDeleted: 'member.action.profile.trait.delete',
19+
MemberSkillsCreated: 'member.action.profile.skills.create',
20+
MemberSkillsUpdated: 'member.action.profile.skills.update'
21+
}
22+
23+
const MAMBO_GET_REWARDS_ALLOWED_FIELDS = [
24+
'awarded.awardedType', 'awarded.message', 'awarded.name', 'awarded.type',
25+
'awarded.reward.active', 'awarded.reward.attrs', 'awarded.reward.id', 'awarded.reward.imageUrl', 'awarded.reward.mimeType', 'awarded.reward.hint', 'awarded.reward.message',
26+
'awardedOn', 'expiryOn', 'isExpired', 'id'
27+
]
28+
29+
const BOOLEAN_OPERATOR = {
30+
AND: 'AND',
31+
OR: 'OR'
32+
}
33+
module.exports = {
34+
ADMIN_ROLES,
35+
SEARCH_BY_EMAIL_ROLES,
36+
AUTOCOMPLETE_ROLES,
37+
EVENT_ORIGINATOR,
38+
EVENT_MIME_TYPE,
39+
TOPICS,
40+
MAMBO_GET_REWARDS_ALLOWED_FIELDS,
41+
BOOLEAN_OPERATOR
42+
}

0 commit comments

Comments
 (0)