|
1 | | -## |
2 | | -## What Does a Delphix Plugin Do? |
3 | | -Delphix is a data management platform that provides the ability to securely copy and share datasets. Using virtualization, you will ingest your data sources and create virtual data copies, which are full read-write capable database instances that use a small fraction of the resources a normal database copy would require. The Delphix engine has built-in support for interfacing with certain types of datasets, such as Oracle, SQL Server and ASE. |
4 | | - |
5 | | -The Delphix virtualization SDK (https://github.com/delphix/virtualization-sdk) provides an interface for building custom data source integrations for the Delphix Dynamic Data Platform. The end users can design/implement a custom plugin which enable them to use custom data source like MySQL, MongoDB, Cassandra, MySQL or something else similar to as if they are using a built-in dataset type with Delphix Engine. |
6 | | - |
7 | | -## MySQL Plugin |
8 | | -MySQL plugin is developed to virtualize MySQL data source leveraging the following built-in MySQL technologies: |
9 | | -Features: |
10 | | - - Environment Discovery: MySQL plugin can discover environments where MySQL server is installed. |
11 | | - - Ingesting Data: Create a dSource using differnt methods specified below. |
12 | | - - VDB Creation: Single node MySQL VDB can be provisioned from the dsource snapshot. |
13 | | - |
14 | | -Different Ways to Ingest Data ( Dsource creation ) |
15 | | - - Replication with Delphix initiated Backup: Delphix takes an initial backup from source DB to ingest data and create a dSource. Delphix also sets up a master-slave replication to keep this dSource in sync with the source database. User can select the databases they want to virtualize |
16 | | - - Replication with User Provided Backup: User provides a backup file from source DB to ingest data and create a dSource. Delphix sets up a master-slave replication to keep this dSource in sync with your source database. |
17 | | - - User Provided Backup with no Replication: User provides a backup file from source DB to ingest data and create a dSource. When a new backup is available, user initiates a resync of the dSource to ingest data from the new backup. |
18 | | - - Manual Backup Ingestion: Delphix creates an empty seed datanase and User manually ingests a backup to create a dSource. |
19 | | - - Simple Tablespace Backup/Subsetting: Allows to create a dSource using a specific list of tables from source database. |
20 | | - |
21 | | - |
22 | | -### Table of Contents |
23 | | -1. [Prerequisites](#requirements-plugin) |
24 | | -2. [Build and Upload Plugin](#upload-plugin) |
25 | | -3. [Download logs](#run_unit_test_case) |
26 | | -4. [Tested Versions](#tested-versions) |
27 | | -5. [Supported Features](#support-features) |
28 | | -6. [Unsupported Features](#unsupported-features) |
29 | | -7. [How to Contribute](#contribute) |
30 | | -8. [Statement of Support](#statement-of-support) |
31 | | -9. [License](#license) |
32 | | - |
33 | | - |
34 | | -### <a id="requirements-plugin"></a>Prerequisites |
35 | | -**Software Requirements** |
36 | | -1. jq - This toolkit requires jq to be configured on Staging and Target Hosts. |
37 | | - |
38 | | -**MySQL instance requirements** |
39 | | -1. Binary logging must be enabled on MySQL source instance. |
40 | | -2. Server ID for the source must be greater than 0. |
41 | | - |
42 | | -**MySQL database user with following privileges** |
43 | | -1. delphixdb |
44 | | -This MySQL user must be configured to have following privilege from the Delphix Engine IP, the staging host IP and localhost |
45 | | -To grant the privilege for this user, use the following command: |
46 | | - |
47 | | -```js |
48 | | -SQL> GRANT SELECT, SHUTDOWN, SUPER, RELOAD, REPLICATION CLIENT,REPLICATION SLAVE,SHOW VIEW, EVENT, TRIGGER on *.* to 'delphix'@'%'; |
49 | | -``` |
50 | | - |
51 | | -OR |
52 | | - |
53 | | -```js |
54 | | -SQL> GRANT ALL PRIVILEGES ON *.* TO '<delphix>'@'%'; |
55 | | -``` |
56 | | - |
57 | | -#### _Staging Host Specific Requirements_ |
58 | | - |
59 | | -**O/S user with following privileges** |
60 | | -1. Regular o/s user. should be able to ps all processes. |
61 | | -2. Should be in the same primary and secondary groups as mysql user ( or the MySQL binary owner ) |
62 | | -3. Execute access on all files within MySQL installation folder - Min permission level 775 recommended. |
63 | | -4. Sudo privileges for mount, umount. See sample below assuming `delphix_os` is used as delphix user. |
64 | | -Example sudoers file entry |
65 | | -```shell |
66 | | -Defaults:delphix_os !requiretty |
67 | | -delphix_os ALL=NOPASSWD: \ |
68 | | -/bin/mount, /bin/umount, /bin/mkdir, /bin/rmdir, /bin/ps |
69 | | -``` |
70 | | - |
71 | | -**Other Staging Host Requirements** |
72 | | - |
73 | | -1. Same version as Source MySQL Binaries installed. |
74 | | -2. A MySQL config file (my.cnf) to be used for the Staging DB instance must be available under Delphix Toolkit Directory. |
75 | | -3. Empty folder on host to hold delphix toolkit [ approximate 2GB free space ] |
76 | | -4. Empty folder on host to mount nfs filesystem. This is just and empty folder with no space requirements and act as base folder for nfs mounts. |
77 | | - |
78 | | - |
79 | | -#### _Target Requirements_ |
80 | | - |
81 | | -**O/S user with following privileges** |
82 | | -1. Regular o/s user. should be able to ps all processes. |
83 | | -2. Should be in the same primary and secondary groups as mysql user ( or the MySQL binary owner ) |
84 | | -3. Execute access on all files within MySQL installation folder - Min permission level 775 recommended. |
85 | | -4. Sudo privileges for mount, umount. See sample below assuming `delphix_os` is used as delphix user. |
86 | | -Example sudoers file entry |
87 | | -```shell |
88 | | -Defaults:delphix_os !requiretty |
89 | | -delphix_os ALL=NOPASSWD: \ |
90 | | -/bin/mount, /bin/umount, /bin/mkdir, /bin/rmdir, /bin/ps |
91 | | -``` |
92 | | - |
93 | | -**Other Staging Host Requirements** |
94 | | - |
95 | | -1. Same version as Source MySQL Binaries installed. |
96 | | -2. A MySQL config file (my.cnf) to be used for the Staging DB instance must be available under Delphix Toolkit Directory. |
97 | | -3. Empty folder on host to hold delphix toolkit [ approximate 2GB free space ] |
98 | | -4. Empty folder on host to mount nfs filesystem. This is just and empty folder with no space requirements and act as base folder for nfs mounts. |
99 | | - |
100 | | - |
101 | | -### <a id="upload-plugin"></a>Steps to build, upload and run unit tests for plugin |
102 | | - |
103 | | - 1. Build the source code. It generates the build with name `artifacts.json`: |
104 | | -```bash |
105 | | - dvp build |
106 | | -``` |
107 | | - |
108 | | - 2. Upload the `artifacts.json` ( generated in step 3 ) on Delphix Engine: |
109 | | -```bash |
110 | | - dvp upload -e <Delphix_Engine_Name> -u <username> --password <password> |
111 | | -``` |
112 | | - |
113 | | - |
114 | | -### <a id="run_unit_test_case"></a>Download plugin logs |
115 | | -#### Plugin Logs: |
116 | | -Download the plugin logs using below command: |
117 | | - |
118 | | -```dvp download-logs -c plugin_config.yml -e <Delphix_Engine_Name> -u admin --password <password>``` |
119 | | - |
120 | | - |
121 | | -### <a id="tested-versions"></a>Tested Versions |
122 | | -- Delphix Engine: 6.0.4.0 and above |
123 | | -- MySQL Version: 5.7.7, 5.7.9, 5.7.10, 5.7.12 |
124 | | -- Linux Version: RHEL 6.x |
125 | | - |
126 | | -### <a id="support-features"></a>Supported Features |
127 | | -- MySQL Replication |
128 | | -- MySQL Tablespace Hotbackup |
129 | | -- MySQL Manual Backup Ingestion |
130 | | - |
131 | | -### <a id="unsupported-features"></a>Unsupported Features |
132 | | -- MySQL Clusters |
133 | | -- Sharded MySQL Databases |
134 | | - |
135 | | - |
136 | | -### <a id="contribute"></a>How to Contribute |
137 | | - |
138 | | -Please read [CONTRIBUTING.md](./CONTRIBUTING.md) to understand the pull requests process. |
139 | | - |
140 | | -### <a id="statement-of-support"></a>Statement of Support |
141 | | - |
142 | | -This software is provided as-is, without warranty of any kind or commercial support through Delphix. See the associated license for additional details. Questions, issues, feature requests, and contributions should be directed to the community as outlined in the [Delphix Community Guidelines](https://delphix.github.io/community-guidelines.html). |
143 | | - |
144 | | -### <a id="license"></a>License |
145 | | - |
146 | | -This is code is licensed under the Apache License 2.0. Full license is available [here](./LICENSE). |
147 | | - |
| 1 | +# msyql_sdk_plugin |
| 2 | +MySQL SDK Plugin for Delphix |
0 commit comments