Skip to content

Commit 55ffcd0

Browse files
authored
Merge pull request #857 from getmaxun/develop
chore: release v0.0.26
2 parents dec45fa + 69ad028 commit 55ffcd0

30 files changed

+1131
-695
lines changed

README.md

Lines changed: 57 additions & 110 deletions
Original file line numberDiff line numberDiff line change
@@ -1,129 +1,76 @@
1-
<h1 align="center">
1+
<h2 align="center">
22
<div>
33
<a href="https://www.maxun.dev/?ref=ghread">
4-
<img src="/src/assets/maxunlogo.png" width="50" />
4+
<img src="/src/assets/maxunlogo.png" width="70" />
55
<br>
66
Maxun
77
</a>
88
</div>
9-
Open-Source No-Code Web Data Extraction Platform <br>
10-
</h1>
9+
The Easiest Way To Extract Web Data With No Code <br>
10+
</h2>
1111

1212
<p align="center">
1313
Maxun lets you train a robot in 2 minutes and scrape the web on auto-pilot. Web data extraction doesn't get easier than this!
1414
</p>
1515

1616

1717
<p align="center">
18-
<a href="https://app.maxun.dev/?ref=ghread"><b>Go To App</b></a> |
19-
<a href="https://docs.maxun.dev/?ref=ghread"><b>Documentation</b></a> |
20-
<a href="https://www.maxun.dev/?ref=ghread"><b>Website</b></a> |
21-
<a href="https://discord.gg/5GbPjBUkws"><b>Discord</b></a> |
22-
<a href="https://x.com/MaxunHQ?ref=ghread"><b>Twitter</b></a> |
18+
<a href="https://app.maxun.dev/?ref=ghread"><b>Go To App</b></a> •
19+
<a href="https://docs.maxun.dev/?ref=ghread"><b>Documentation</b></a> •
20+
<a href="https://www.maxun.dev/?ref=ghread"><b>Website</b></a> •
21+
<a href="https://discord.gg/5GbPjBUkws"><b>Discord</b></a> •
2322
<a href="https://www.youtube.com/@MaxunOSS?ref=ghread"><b>Watch Tutorials</b></a>
2423
<br />
2524
<br />
2625
<a href="https://trendshift.io/repositories/12113" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12113" alt="getmaxun%2Fmaxun | Trendshift" style="width: 250px; height: 55px; margin-top: 10px;" width="250" height="55"/></a>
2726
</p>
2827

29-
![maxun_gif](https://github.com/user-attachments/assets/3e0b0cf8-9e52-44d2-a140-b26b7b481477)
30-
31-
<img src="https://static.scarf.sh/a.png?x-pxid=c12a77cc-855e-4602-8a0f-614b2d0da56a" />
32-
33-
# Getting Started
34-
The simplest & fastest way to get started is to use the hosted version: https://app.maxun.dev. Maxun Cloud deals with anti-bot detection, huge proxy network with automatic proxy rotation, and CAPTCHA solving.
35-
36-
# Local Installation
37-
1. Create a root folder for your project (e.g. 'maxun')
38-
2. Create a file named `.env` in the root folder of the project
39-
3. Example env file can be viewed [here](https://github.com/getmaxun/maxun/blob/master/ENVEXAMPLE). Copy all content of example env to your `.env` file.
40-
4. Choose your installation method below
41-
42-
### Docker Compose
43-
1. Copy paste the [docker-compose.yml file](https://github.com/getmaxun/maxun/blob/master/docker-compose.yml) into your root folder
44-
2. Ensure you have setup the `.env` file in that same folder
45-
3. Run the command below from a terminal
46-
```
47-
docker-compose up -d
48-
```
49-
You can access the frontend at http://localhost:5173/ and backend at http://localhost:8080/
50-
51-
### Without Docker
52-
1. Ensure you have Node.js, PostgreSQL, MinIO and Redis installed on your system.
53-
2. Run the commands below
54-
```
55-
git clone https://github.com/getmaxun/maxun
56-
57-
# change directory to the project root
58-
cd maxun
59-
60-
# install dependencies
61-
npm install
62-
63-
# change directory to maxun-core to install dependencies
64-
cd maxun-core
65-
npm install
66-
67-
# get back to the root directory
68-
cd ..
69-
70-
# install chromium and its dependencies
71-
npx playwright install --with-deps chromium
72-
73-
# get back to the root directory
74-
cd ..
75-
76-
# start frontend and backend together
77-
npm run start
78-
```
79-
You can access the frontend at http://localhost:5173/ and backend at http://localhost:8080/
80-
81-
82-
# Environment Variables
83-
1. Create a file named `.env` in the root folder of the project
84-
2. Example env file can be viewed [here](https://github.com/getmaxun/maxun/blob/master/ENVEXAMPLE).
85-
86-
| Variable | Mandatory | Description | If Not Set |
87-
|-----------------------|-----------|----------------------------------------------------------------------------------------------|--------------------------------------------------------------|
88-
| `BACKEND_PORT` | Yes | Port to run backend on. Needed for Docker setup | Default value: 8080 |
89-
| `FRONTEND_PORT` | Yes | Port to run frontend on. Needed for Docker setup | Default value: 5173 |
90-
| `BACKEND_URL` | Yes | URL to run backend on. | Default value: http://localhost:8080 |
91-
| `VITE_BACKEND_URL` | Yes | URL used by frontend to connect to backend | Default value: http://localhost:8080 |
92-
| `PUBLIC_URL` | Yes | URL to run frontend on. | Default value: http://localhost:5173 |
93-
| `VITE_PUBLIC_URL` | Yes | URL used by backend to connect to frontend | Default value: http://localhost:5173 |
94-
| `JWT_SECRET` | Yes | Secret key used to sign and verify JSON Web Tokens (JWTs) for authentication. | JWT authentication will not work. |
95-
| `DB_NAME` | Yes | Name of the Postgres database to connect to. | Database connection will fail. |
96-
| `DB_USER` | Yes | Username for Postgres database authentication. | Database connection will fail. |
97-
| `DB_PASSWORD` | Yes | Password for Postgres database authentication. | Database connection will fail. |
98-
| `DB_HOST` | Yes | Host address where the Postgres database server is running. | Database connection will fail. |
99-
| `DB_PORT` | Yes | Port number used to connect to the Postgres database server. | Database connection will fail. |
100-
| `ENCRYPTION_KEY` | Yes | Key used for encrypting sensitive data (proxies, passwords). | Encryption functionality will not work. |
101-
| `SESSION_SECRET` | No | A strong, random string used to sign session cookies | Uses default secret. Recommended to define your own session secret to avoid session hijacking. |
102-
| `MINIO_ENDPOINT` | Yes | Endpoint URL for MinIO, to store Robot Run Screenshots. | Connection to MinIO storage will fail. |
103-
| `MINIO_PORT` | Yes | Port number for MinIO service. | Connection to MinIO storage will fail. |
104-
| `MINIO_CONSOLE_PORT` | No | Port number for MinIO WebUI service. Needed for Docker setup. | Cannot access MinIO Web UI. |
105-
| `MINIO_ACCESS_KEY` | Yes | Access key for authenticating with MinIO. | MinIO authentication will fail. |
106-
| `GOOGLE_CLIENT_ID` | No | Client ID for Google OAuth. Used for Google Sheet integration authentication. | Google login will not work. |
107-
| `GOOGLE_CLIENT_SECRET`| No | Client Secret for Google OAuth. Used for Google Sheet integration authentication. | Google login will not work. |
108-
| `GOOGLE_REDIRECT_URI` | No | Redirect URI for handling Google OAuth responses. | Google login will not work. |
109-
| `AIRTABLE_CLIENT_ID` | No | Client ID for Airtable, used for Airtable integration authentication. | Airtable login will not work. |
110-
| `AIRTABLE_REDIRECT_URI` | No | Redirect URI for handling Airtable OAuth responses. | Airtable login will not work. |
111-
| `MAXUN_TELEMETRY` | No | Disables telemetry to stop sending anonymous usage data. Keeping it enabled helps us understand how the product is used and assess the impact of any new changes. Please keep it enabled. | Telemetry data will not be collected. |
112-
113-
# How Do I Self-Host?
114-
Checkout community self hosting guide: https://docs.maxun.dev/self-host
115-
116-
# How Does It Work?
117-
Maxun lets you create custom robots which emulate user actions and extract data. A robot can perform any of the actions: <b>Capture List, Capture Text or Capture Screenshot. Once a robot is created, it will keep extracting data for you without manual intervention</b>
118-
119-
![Screenshot 2024-10-23 222138](https://github.com/user-attachments/assets/53573c98-769e-490d-829e-ada9fac0764f)
120-
121-
## 1. Robot Actions
122-
1. Capture List: Useful to extract structured and bulk items from the website. Example: Scrape products from Amazon etc.
28+
https://github.com/user-attachments/assets/c6baa75f-b950-482c-8d26-8a8b6c5382c3
29+
30+
### Getting Started
31+
The simplest & fastest way to get started is to use the hosted version: https://app.maxun.dev. You can self-host if you like!
32+
33+
### Installation
34+
Maxun can run locally with or without Docker
35+
1. [Setup with Docker Compose](https://docs.maxun.dev/installation/docker)
36+
2. [Setup without Docker](https://docs.maxun.dev/installation/local)
37+
3. [Environment Variables](https://docs.maxun.dev/installation/environment_variables)
38+
39+
### Upgrading & Self Hosting
40+
1. [Self Host Maxun With Docker & Portainer](https://docs.maxun.dev/self-host)
41+
2. [Upgrade Maxun With Docker Compose Setup](https://docs.maxun.dev/installation/upgrade#upgrading-with-docker-compose)
42+
3. [Upgrade Maxun Without Docker Compose Setup](https://docs.maxun.dev/installation/upgrade#upgrading-with-local-setup)
43+
44+
### How Does It Work?
45+
Maxun lets you create custom robots which emulate user actions and extract data. A robot can perform any of the actions: Capture List, Capture Text or Capture Screenshot. Once a robot is created, it will keep extracting data for you without manual intervention.
46+
1. Capture List: Useful to extract structured and bulk items from the website.
12347
2. Capture Text: Useful to extract individual text content from the website.
12448
3. Capture Screenshot: Get fullpage or visible section screenshots of the website.
12549

126-
# Features
50+
### Sponsors
51+
<table>
52+
<tr>
53+
<td width="229">
54+
<br/>
55+
<a href="https://www.lambdatest.com/?utm_source=maxun&utm_medium=sponsor" target="_blank">
56+
<img src="https://github.com/user-attachments/assets/904dd40e-0498-47dd-98f1-7fa6d318adb9" /><br/><br/>
57+
<b>LambdaTest</b>
58+
</a>
59+
<br/>
60+
<sub>GenAI-powered Quality Engineering Platform that empowers teams to test intelligently, smarter, and ship faster.</sub>
61+
</td>
62+
<td width="250">
63+
<a href="https://app.cyberyozh.com/?utm_source=github&utm_medium=maxun" target="_blank">
64+
<img src="https://github.com/user-attachments/assets/c0ae7929-003a-4e1e-b23b-d174ac0aba4f" /><br/>
65+
<b>CyberYozh App</b>
66+
</a>
67+
<br/>
68+
<sub>Infrastructure for developers working with multi‑accounting & automation in one place.</sub>
69+
</td>
70+
</tr>
71+
</table>
72+
73+
### Features
12774
- ✨ Extract Data With No-Code
12875
- ✨ Handle Pagination & Scrolling
12976
- ✨ Run Robots On A Specific Schedule
@@ -134,11 +81,11 @@ Maxun lets you create custom robots which emulate user actions and extract data.
13481
- ✨ Integrations
13582
- ✨ MCP
13683

137-
# Use Cases
84+
### Use Cases
13885
Maxun can be used for various use-cases, including lead generation, market research, content aggregation and more.
13986
View use-cases in detail here: https://www.maxun.dev/#usecases
14087

141-
# Screenshots
88+
### Screenshots
14289
![Maxun PH Launch (1)-1-1](https://github.com/user-attachments/assets/d7c75fa2-2bbc-47bb-a5f6-0ee6c162f391)
14390
![Maxun PH Launch (1)-2-1](https://github.com/user-attachments/assets/d85a3ec7-8ce8-4daa-89aa-52d9617e227a)
14491
![Maxun PH Launch (1)-3-1](https://github.com/user-attachments/assets/4bd5a0b4-485d-44f4-a487-edd9afc18b11)
@@ -149,18 +96,18 @@ View use-cases in detail here: https://www.maxun.dev/#usecases
14996
![Maxun PH Launch (1)-8-1](https://github.com/user-attachments/assets/16ee4a71-772a-49ae-a0e5-cb0529519bda)
15097
![Maxun PH Launch (1)-9-1](https://github.com/user-attachments/assets/160f46fa-0357-4c1b-ba50-b4fe64453bb7)
15198

152-
# Note
99+
### Note
153100
This project is in early stages of development. Your feedback is very important for us - we're actively working on improvements. </a>
154101

155-
# License
102+
### License
156103
<p>
157104
This project is licensed under <a href="./LICENSE">AGPLv3</a>.
158105
</p>
159106

160-
# Support Us
107+
### Support Us
161108
Star the repository, contribute if you love what we’re building, or [sponsor us](https://github.com/sponsors/amhsirak).
162109

163-
# Contributors
110+
### Contributors
164111
Thank you to the combined efforts of everyone who contributes!
165112

166113
<a href="https://github.com/getmaxun/maxun/graphs/contributors">

SETUP.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# Local Installation
2+
1. Create a root folder for your project (e.g. 'maxun')
3+
2. Create a file named `.env` in the root folder of the project
4+
3. Example env file can be viewed [here](https://github.com/getmaxun/maxun/blob/master/ENVEXAMPLE). Copy all content of example env to your `.env` file.
5+
4. Choose your installation method below
6+
7+
### Docker Compose
8+
1. Copy paste the [docker-compose.yml file](https://github.com/getmaxun/maxun/blob/master/docker-compose.yml) into your root folder
9+
2. Ensure you have setup the `.env` file in that same folder
10+
3. Run the command below from a terminal
11+
```
12+
docker-compose up -d
13+
```
14+
You can access the frontend at http://localhost:5173/ and backend at http://localhost:8080/
15+
16+
### Without Docker
17+
1. Ensure you have Node.js, PostgreSQL, MinIO and Redis installed on your system.
18+
2. Run the commands below
19+
```
20+
git clone https://github.com/getmaxun/maxun
21+
22+
# change directory to the project root
23+
cd maxun
24+
25+
# install dependencies
26+
npm install
27+
28+
# change directory to maxun-core to install dependencies
29+
cd maxun-core
30+
npm install
31+
32+
# get back to the root directory
33+
cd ..
34+
35+
# install chromium and its dependencies
36+
npx playwright install --with-deps chromium
37+
38+
# get back to the root directory
39+
cd ..
40+
41+
# start frontend and backend together
42+
npm run start
43+
```
44+
You can access the frontend at http://localhost:5173/ and backend at http://localhost:8080/
45+
46+
47+
# Environment Variables
48+
1. Create a file named `.env` in the root folder of the project
49+
2. Example env file can be viewed [here](https://github.com/getmaxun/maxun/blob/master/ENVEXAMPLE).
50+
51+
| Variable | Mandatory | Description | If Not Set |
52+
|-----------------------|-----------|----------------------------------------------------------------------------------------------|--------------------------------------------------------------|
53+
| `BACKEND_PORT` | Yes | Port to run backend on. Needed for Docker setup | Default value: 8080 |
54+
| `FRONTEND_PORT` | Yes | Port to run frontend on. Needed for Docker setup | Default value: 5173 |
55+
| `BACKEND_URL` | Yes | URL to run backend on. | Default value: http://localhost:8080 |
56+
| `VITE_BACKEND_URL` | Yes | URL used by frontend to connect to backend | Default value: http://localhost:8080 |
57+
| `PUBLIC_URL` | Yes | URL to run frontend on. | Default value: http://localhost:5173 |
58+
| `VITE_PUBLIC_URL` | Yes | URL used by backend to connect to frontend | Default value: http://localhost:5173 |
59+
| `JWT_SECRET` | Yes | Secret key used to sign and verify JSON Web Tokens (JWTs) for authentication. | JWT authentication will not work. |
60+
| `DB_NAME` | Yes | Name of the Postgres database to connect to. | Database connection will fail. |
61+
| `DB_USER` | Yes | Username for Postgres database authentication. | Database connection will fail. |
62+
| `DB_PASSWORD` | Yes | Password for Postgres database authentication. | Database connection will fail. |
63+
| `DB_HOST` | Yes | Host address where the Postgres database server is running. | Database connection will fail. |
64+
| `DB_PORT` | Yes | Port number used to connect to the Postgres database server. | Database connection will fail. |
65+
| `ENCRYPTION_KEY` | Yes | Key used for encrypting sensitive data (proxies, passwords). | Encryption functionality will not work. |
66+
| `SESSION_SECRET` | No | A strong, random string used to sign session cookies | Uses default secret. Recommended to define your own session secret to avoid session hijacking. |
67+
| `MINIO_ENDPOINT` | Yes | Endpoint URL for MinIO, to store Robot Run Screenshots. | Connection to MinIO storage will fail. |
68+
| `MINIO_PORT` | Yes | Port number for MinIO service. | Connection to MinIO storage will fail. |
69+
| `MINIO_CONSOLE_PORT` | No | Port number for MinIO WebUI service. Needed for Docker setup. | Cannot access MinIO Web UI. |
70+
| `MINIO_ACCESS_KEY` | Yes | Access key for authenticating with MinIO. | MinIO authentication will fail. |
71+
| `GOOGLE_CLIENT_ID` | No | Client ID for Google OAuth. Used for Google Sheet integration authentication. | Google login will not work. |
72+
| `GOOGLE_CLIENT_SECRET`| No | Client Secret for Google OAuth. Used for Google Sheet integration authentication. | Google login will not work. |
73+
| `GOOGLE_REDIRECT_URI` | No | Redirect URI for handling Google OAuth responses. | Google login will not work. |
74+
| `AIRTABLE_CLIENT_ID` | No | Client ID for Airtable, used for Airtable integration authentication. | Airtable login will not work. |
75+
| `AIRTABLE_REDIRECT_URI` | No | Redirect URI for handling Airtable OAuth responses. | Airtable login will not work. |
76+
| `MAXUN_TELEMETRY` | No | Disables telemetry to stop sending anonymous usage data. Keeping it enabled helps us understand how the product is used and assess the impact of any new changes. Please keep it enabled. | Telemetry data will not be collected. |

maxun-core/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "maxun-core",
3-
"version": "0.0.25",
3+
"version": "0.0.26",
44
"description": "Core package for Maxun, responsible for data extraction",
55
"main": "build/index.js",
66
"typings": "build/index.d.ts",

0 commit comments

Comments
 (0)