|
1 | | -repex |
2 | | -===== |
| 1 | +python-packer |
| 2 | +============= |
| 3 | + |
| 4 | +A Python interface for executing `Packer <http://www.packer.io>`_. |
3 | 5 |
|
4 | 6 | - Master Branch |Build Status| |
5 | 7 | - PyPI |PyPI| |
6 | 8 | - Version |PypI| |
7 | 9 |
|
8 | | -``repex`` replaces strings in single/multiple files based on regular |
9 | | -expressions. |
10 | | - |
11 | | -Why not use sed you ask? Because ``repex`` provides some layers of |
12 | | -protection and an easy to use config yaml in which you easily add new |
13 | | -files and folders to iterate through. |
14 | | - |
15 | | -The layers are: \* Match and only then replace in the matched regular |
16 | | -expression. \* Check for existing strings in a file before replacing |
17 | | -anything. \* Exclude files and folders so that you don't screw up. |
18 | | - |
19 | | -AND, you can use variables (sorta jinja2 style). How cool is that? See |
20 | | -reference config below. |
21 | | - |
22 | | -Installation |
23 | | -~~~~~~~~~~~~ |
24 | | - |
25 | | -.. code:: shell |
26 | | -
|
27 | | - pip install repex |
28 | | -
|
29 | | -For dev: |
30 | | - |
31 | | -.. code:: shell |
32 | | -
|
33 | | - pip install https://github.com/cloudify-cosmo/repex/archive/master.tar.gz |
34 | | -
|
35 | | -Usage |
36 | | -~~~~~ |
37 | | - |
38 | | -Let's say you have files named "VERSION" in different directories which |
39 | | -look like this: |
40 | | - |
41 | | -.. code:: json |
42 | | -
|
43 | | - { |
44 | | - "date": "", |
45 | | - "commit": "", |
46 | | - "version": "3.1.0-m2", |
47 | | - "version_other": "3.1.2-m1", |
48 | | - "build": "8" |
49 | | - } |
50 | | -
|
51 | | -And you'd like to replace 3.1.0-m2 with 3.1.0-m3 in all of those files |
52 | | - |
53 | | -You would create a repex config.yaml with the following: |
54 | | - |
55 | | -.. code:: yaml |
56 | | -
|
57 | | -
|
58 | | - variables: |
59 | | - base_dir: . |
60 | | -
|
61 | | - paths: |
62 | | - - type: VERSION |
63 | | - path: resources |
64 | | - excluded: |
65 | | - - excluded_file.file |
66 | | - base_directory: "{{ .base_dir }}" |
67 | | - match: '"version": "\d+\.\d+(\.\d+)?(-\w\d+)?' |
68 | | - replace: \d+\.\d+(\.\d+)?(-\w\d+)? |
69 | | - with: "{{ .version }}" |
70 | | - validate_before: true |
71 | | - must_include: |
72 | | - - date |
73 | | - - commit |
74 | | - - version |
75 | | - validator: |
76 | | - type: per_file |
77 | | - path: my/validator/script/path.py |
78 | | - function: my_validation_function |
79 | | -
|
80 | | -and do the following |
81 | | - |
82 | | -.. code:: python |
83 | | -
|
84 | | -
|
85 | | - import os |
86 | | - import repex.repex as rpx |
87 | | -
|
88 | | - CONFIG_YAML_FILE = "tester.yaml" |
89 | | - VERSION = os.environ['VERSION'] # '3.1.0-m3' |
90 | | -
|
91 | | - variables = { |
92 | | - 'version': VERSION, |
93 | | - } |
94 | | -
|
95 | | - rpx.iterate(CONFIG_YAML_FILE, variables) |
96 | | -
|
97 | | -and even add a validator file: |
98 | | - |
99 | | -.. code:: python |
100 | | -
|
101 | | -
|
102 | | - def my_validation_function(version_file_path): |
103 | | - result = verify_replacement() |
104 | | - return True if result else False |
105 | | -
|
106 | | -Config yaml Explained |
107 | | -^^^^^^^^^^^^^^^^^^^^^ |
108 | | - |
109 | | -IMPORTANT NOTE: variables MUST be enclosed within single or double |
110 | | -quotes or they will not expand! Might fix that in future versions... |
111 | | - |
112 | | -ANOTHER IMPORTANT NOTE: variables must be structured EXACTLY like this: |
113 | | -{{ .VER\_NAME }} Don't forget the spaces! |
114 | | - |
115 | | -- ``variables`` is a dict of variables you can use throughout the |
116 | | - config. ``type``, ``path``, ``base_directory``, ``match``, |
117 | | - ``replace`` and ``with`` can all receive variables. For now, all |
118 | | - attributes which are not strings cannot receive variables. This might |
119 | | - change in future versions. ``variables`` can be sent to one of the 3 |
120 | | - basic functions described below or by being hardcoded into the yaml. |
121 | | - Variables with the same name sent via the API will override the |
122 | | - hardcoded ones. |
123 | | -- ``type`` is a regex string representing the file name you're looking |
124 | | - for. |
125 | | -- ``path`` is a regex string representing the path in which you'd like |
126 | | - to search for files (so, for instance, if you only want to replace |
127 | | - files in directory names starting with "my-", you would write |
128 | | - "my-.\*"). If ``path`` is a path to a single file, the ``type`` |
129 | | - attribute must not be configured. |
130 | | -- ``excluded`` is a list of excluded paths. The paths must be relative |
131 | | - to the working directory, NOT to the ``path`` variable. |
132 | | -- ``base_directory`` is the directory from which you'd like to start |
133 | | - the recursive search for files. If ``path`` is a path to a file, this |
134 | | - property can be omitted. Alternatively, you can set the |
135 | | - ``base_directory`` and a ``path`` relative to it. |
136 | | -- ``match`` is the initial regex based string you'd like to match |
137 | | - before replacing the expression. This provides a more robust way to |
138 | | - replace strings where you first match the exact area in which you'd |
139 | | - like to replace the expression and only then match the expression you |
140 | | - want to replace within it. It also provides a way to replace only |
141 | | - specific instances of an expression, and not all. |
142 | | -- ``replace`` - which regex would you like to replace? |
143 | | -- ``with`` - what you replace with. |
144 | | -- ``validate_before`` - a flag stating that you'd like to validate that |
145 | | - the pattern you're looking for exists in the file and that all |
146 | | - strings in ``must_include`` exists in the file as well. |
147 | | -- ``must_include`` - as an additional layer of security, you can |
148 | | - specify a set of regex based strings to look for to make sure that |
149 | | - the files you're dealing with are the actual files you'd like to |
150 | | - replace the expressions in. |
151 | | -- ``validator`` - validator allows you to run a validation function |
152 | | - after replacing expressions. It receives ``type`` which can be either |
153 | | - ``per_file`` or ``per_type`` where ``per_file`` runs the validation |
154 | | - on every file while ``per_type`` runs once for every ``type`` of |
155 | | - file; it receives a ``path`` to the script and a ``function`` within |
156 | | - the script to call. Note that each validation function must return |
157 | | - ``True`` if successful and ``False`` if failed. The validating |
158 | | - function receives the file's path as a parameter. |
159 | | - |
160 | | -In case you're providing a path to a file rather than a directory: |
161 | | - |
162 | | -- ``type`` and ``base_directory`` are depracated |
163 | | -- you can provide a ``to_file`` key with the path to the file you'd |
164 | | - like to create after replacing. |
165 | | - |
166 | | -Basic Functions |
167 | | -^^^^^^^^^^^^^^^ |
168 | | - |
169 | | -3 basic functions are provided: |
170 | | - |
171 | | -The following examples all perform the exact same function (``iterate``) |
172 | | -but using the different provided methods for the sake of granularity. |
173 | | - |
174 | | -Note that under normal circumstanaces, you will not need to drill down |
175 | | -into these and just use ``iterate``. |
176 | | - |
177 | | -iterate |
178 | | -''''''' |
179 | | - |
180 | | -Receives the config yaml file and the variables dict and iterates |
181 | | -through the config file's ``paths`` list destroying everything that |
182 | | -comes in its path :) |
183 | | - |
184 | | -.. code:: python |
185 | | -
|
186 | | -
|
187 | | - import os |
188 | | - import repex.repex as rpx |
189 | | -
|
190 | | - CONFIG_YAML_FILE = "tester.yaml" |
191 | | - VERSION = os.environ['VERSION'] # '3.1.0-m3' |
192 | | - VERBOSE = True |
193 | | -
|
194 | | - variables = { |
195 | | - 'version': VERSION, |
196 | | - 'base_dir': . |
197 | | - } |
198 | | -
|
199 | | - rpx.iterate(CONFIG_YAML_FILE, variables, verbose=VERBOSE) |
200 | | -
|
201 | | -handle\_path |
202 | | -'''''''''''' |
203 | | - |
204 | | -Receives one of the objects in the ``paths`` list in the config yaml |
205 | | -file and the variables dict, finds all files of name ``type`` and |
206 | | -processes them (is used by ``iterate``). |
207 | | - |
208 | | -.. code:: python |
209 | | -
|
210 | | -
|
211 | | - import os |
212 | | - import repex.repex as rpx |
213 | | -
|
214 | | - CONFIG_YAML_FILE = "tester.yaml" |
215 | | - VERSION = os.environ['VERSION'] # '3.1.0-m3' |
216 | | - VERBOSE = True |
217 | | -
|
218 | | - variables = { |
219 | | - 'version': VERSION, |
220 | | - 'base_dir': . |
221 | | - } |
222 | | -
|
223 | | - # this is what iterate would do if it was called directly |
224 | | - config = rpx.import_config(CONFIG_YAML_FILE) |
225 | | - vars = config.get('variables', {}) |
226 | | - vars.update(variables) |
227 | | - for p in config['paths']: |
228 | | - rpx.handle_path(p, vars, verbose=VERBOSE) |
229 | | -
|
230 | | -handle\_file |
231 | | -'''''''''''' |
232 | | - |
233 | | -Receives one of the objects in the ``paths`` list in the config yaml |
234 | | -file and the variables dict, and processes the specific file specified |
235 | | -in the ``path`` key (used by ``handle_path``). |
236 | | - |
237 | | -IMPORTANT: |
238 | | - |
239 | | -- Variable expansion occurs only in ``handle_path``. Therefore, if |
240 | | - variables exist, we must manually call the variable expansion method. |
241 | | -- The ``path`` attribute in each object must be a path to a file. |
242 | | -- ``get_all_files`` will find all files with name ``type`` in ``path`` |
243 | | - from dir ``base_directory``, excluding ``excluded``. |
244 | | - |
245 | | -.. code:: python |
246 | | -
|
247 | | -
|
248 | | - import os |
249 | | - import repex.repex as rpx |
250 | | -
|
251 | | -
|
252 | | - CONFIG_YAML_FILE = "tester.yaml" |
253 | | - VERSION = os.environ['VERSION'] # '3.1.0-m3' |
254 | | - VERBOSE = True |
255 | | -
|
256 | | - variables = { |
257 | | - 'version': VERSION, |
258 | | - 'base_dir': . |
259 | | - } |
260 | | -
|
261 | | - # this is what iterate would do if it was called directly |
262 | | - config = rpx.import_config(CONFIG_YAML_FILE) |
263 | | - vars = config.get('variables', {}) |
264 | | - vars.update(variables) |
265 | | - for p in config['paths']: |
266 | | - files = get_all_files( |
267 | | - p['type'], p['path'], p['base_directory'], p['excluded'], , verbose=VERBOSE) |
268 | | - # this will run the validator if applicable. |
269 | | - _validate(p['path']) |
270 | | - # this is what handle_path would do if it was called directly |
271 | | - var_expander = rpx.VarHandler(p) |
272 | | - p = var_expander.expand(variables) |
273 | | - for file in files: |
274 | | - p['path'] = file |
275 | | - rpx.handle_file(file, vars, verbose=VERBOSE) |
276 | | -
|
277 | | -.. |Build Status| image:: https://travis-ci.org/cloudify-cosmo/repex.svg?branch=master |
278 | | - :target: https://travis-ci.org/cloudify-cosmo/repex |
279 | | -.. |PyPI| image:: http://img.shields.io/pypi/dm/repex.svg |
280 | | - :target: http://img.shields.io/pypi/dm/repex.svg |
281 | | -.. |PypI| image:: http://img.shields.io/pypi/v/repex.svg |
282 | | - :target: http://img.shields.io/pypi/v/repex.svg |
| 10 | +.. |Build Status| image:: https://travis-ci.org/nir0s/python-packer.svg?branch=master |
| 11 | + :target: https://travis-ci.org/nir0s/python-packer |
| 12 | +.. |PyPI| image:: http://img.shields.io/pypi/dm/python-packer.svg |
| 13 | + :target: http://img.shields.io/pypi/dm/python-packer.svg |
| 14 | +.. |PypI| image:: http://img.shields.io/pypi/v/python-packer.svg |
| 15 | + :target: http://img.shields.io/pypi/v/python-packer.svg |
0 commit comments