Skip to content

Commit cf62a83

Browse files
atollkatollk
authored
Merge branch 'master' into issue_450
2 parents 01f2d34 + 7fd6558 commit cf62a83

34 files changed

+1341
-331
lines changed

‎.github/workflows/package.yml‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
name: Package
2+
3+
on:
4+
- push
5+
- pull_request
6+
7+
jobs:
8+
9+
build-wheel:
10+
runs-on: ubuntu-latest
11+
name: Build wheel distribution
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v2
15+
with:
16+
submodules: true
17+
- name: Setup Python ${{ matrix.python-version }}
18+
uses: actions/setup-python@v2
19+
with:
20+
python-version: ${{ matrix.python-version }}
21+
- name: Update build dependencies
22+
run: python -m pip install -U pip wheel setuptools
23+
- name: Build wheel distribution
24+
run: python setup.py bdist_wheel
25+
- name: Store built wheel
26+
uses: actions/upload-artifact@v2
27+
with:
28+
name: dist
29+
path: dist/*
30+
31+
build-sdist:
32+
runs-on: ubuntu-latest
33+
name: Build source distribution
34+
steps:
35+
- name: Checkout code
36+
uses: actions/checkout@v2
37+
with:
38+
submodules: true
39+
- name: Set up Python 3.9
40+
uses: actions/setup-python@v2
41+
with:
42+
python-version: 3.9
43+
- name: Update build dependencies
44+
run: python -m pip install -U pip wheel setuptools
45+
- name: Build source distribution
46+
run: python setup.py sdist
47+
- name: Store source distribution
48+
uses: actions/upload-artifact@v2
49+
with:
50+
name: dist
51+
path: dist/*
52+
53+
test-sdist:
54+
runs-on: ubuntu-latest
55+
name: Test source distribution
56+
needs:
57+
- build-sdist
58+
steps:
59+
- name: Checkout code
60+
uses: actions/checkout@v2
61+
with:
62+
submodules: true
63+
- name: Setup Python 3.9
64+
uses: actions/setup-python@v2
65+
with:
66+
python-version: 3.9
67+
- name: Download source distribution
68+
uses: actions/download-artifact@v2
69+
with:
70+
name: dist
71+
path: dist
72+
- name: Install source distribution
73+
run: python -m pip install dist/fs-*.tar.gz
74+
- name: Remove source code
75+
run: rm -rvd fs
76+
- name: Install test requirements
77+
run: python -m pip install -r tests/requirements.txt
78+
- name: Test installed package
79+
run: python -m unittest discover -vv
80+
81+
test-wheel:
82+
runs-on: ubuntu-latest
83+
name: Test wheel distribution
84+
needs:
85+
- build-wheel
86+
steps:
87+
- name: Checkout code
88+
uses: actions/checkout@v2
89+
with:
90+
submodules: true
91+
- name: Setup Python 3.9
92+
uses: actions/setup-python@v2
93+
with:
94+
python-version: 3.9
95+
- name: Download wheel distribution
96+
uses: actions/download-artifact@v2
97+
with:
98+
name: dist
99+
path: dist
100+
- name: Install source distribution
101+
run: python -m pip install dist/fs-*.whl
102+
- name: Remove source code
103+
run: rm -rvd fs
104+
- name: Install test requirements
105+
run: python -m pip install -r tests/requirements.txt
106+
- name: Test installed package
107+
run: python -m unittest discover -vv
108+
109+
upload:
110+
environment: PyPI
111+
runs-on: ubuntu-latest
112+
name: Upload
113+
needs:
114+
- build-sdist
115+
- build-wheel
116+
- test-sdist
117+
- test-wheel
118+
steps:
119+
- name: Download built distributions
120+
uses: actions/download-artifact@v2
121+
with:
122+
name: dist
123+
path: dist
124+
- name: Publish distributions to PyPI
125+
if: startsWith(github.ref, 'refs/tags/v')
126+
uses: pypa/gh-action-pypi-publish@master
127+
with:
128+
user: __token__
129+
password: ${{ secrets.PYPI_API_TOKEN }}
130+
skip_existing: false

‎.github/workflows/test.yml‎

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,20 +32,36 @@ jobs:
3232
run: python -m pip install tox tox-gh-actions
3333
- name: Test with tox
3434
run: python -m tox
35-
- name: Collect coverage results
36-
uses: AndreMiras/coveralls-python-action@develop
35+
- name: Store partial coverage reports
36+
uses: actions/upload-artifact@v2
3737
with:
38-
parallel: true
39-
flag-name: test (${{ matrix.python-version}})
38+
name: coverage
39+
path: .coverage.*
4040

4141
coveralls:
4242
needs: test
4343
runs-on: ubuntu-latest
4444
steps:
45+
- name: Checkout code
46+
uses: actions/checkout@v1
47+
- name: Setup Python 3.9
48+
uses: actions/setup-python@v2
49+
with:
50+
python-version: 3.9
51+
- name: Install coverage package
52+
run: python -m pip install -U coverage
53+
- name: Download partial coverage reports
54+
uses: actions/download-artifact@v2
55+
with:
56+
name: coverage
57+
- name: Combine coverage
58+
run: python -m coverage combine
59+
- name: Report coverage
60+
run: python -m coverage report
61+
- name: Export coverage to XML
62+
run: python -m coverage xml
4563
- name: Upload coverage statistics to Coveralls
4664
uses: AndreMiras/coveralls-python-action@develop
47-
with:
48-
parallel-finished: true
4965

5066
lint:
5167
runs-on: ubuntu-latest

‎CHANGELOG.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,24 +13,46 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
1313
- Added FTP over TLS (FTPS) support to FTPFS.
1414
Closes [#437](https://github.com/PyFilesystem/pyfilesystem2/issues/437),
1515
[#449](https://github.com/PyFilesystem/pyfilesystem2/pull/449).
16+
- `PathError` now supports wrapping an exception using the `exc` argument.
17+
Closes [#453](https://github.com/PyFilesystem/pyfilesystem2/issues/453).
18+
- Better documentation of the `writable` parameter of `fs.open_fs`, and
19+
hint about using `fs.wrap.read_only` when a read-only filesystem is
20+
required. Closes [#441](https://github.com/PyFilesystem/pyfilesystem2/issues/441).
1621

1722
### Changed
1823

1924
- Make `FS.upload` explicit about the expected error when the parent directory of the destination does not exist.
2025
Closes [#445](https://github.com/PyFilesystem/pyfilesystem2/pull/445).
2126
- Migrate continuous integration from Travis-CI to GitHub Actions and introduce several linters
2227
again in the build steps ([#448](https://github.com/PyFilesystem/pyfilesystem2/pull/448)).
23-
Closes [#446](https://github.com/PyFilesystem/pyfilesystem2/pull/446).
28+
Closes [#446](https://github.com/PyFilesystem/pyfilesystem2/issues/446).
2429
- Stop requiring `pytest` to run tests, allowing any test runner supporting `unittest`-style
2530
test suites.
2631
- `FSTestCases` now builds the large data required for `upload` and `download` tests only
2732
once in order to reduce the total testing time.
33+
- `MemoryFS.move` and `MemoryFS.movedir` will now avoid copying data.
34+
Closes [#452](https://github.com/PyFilesystem/pyfilesystem2/issues/452).
35+
- `FS.removetree("/")` behaviour has been standardized in all filesystems, and
36+
is expected to clear the contents of the root folder without deleting it.
37+
Closes [#471](https://github.com/PyFilesystem/pyfilesystem2/issues/471).
38+
- `FS.getbasic` is now deprecated, as it is redundant with `FS.getinfo`,
39+
and `FS.getinfo` is now explicitly expected to return the *basic* info
40+
namespace unconditionally. Closes [#469](https://github.com/PyFilesystem/pyfilesystem2/issues/469).
2841

2942
### Fixed
3043

3144
- Make `FTPFile`, `MemoryFile` and `RawWrapper` accept [`array.array`](https://docs.python.org/3/library/array.html)
3245
arguments for the `write` and `writelines` methods, as expected by their base class [`io.RawIOBase`](https://docs.python.org/3/library/io.html#io.RawIOBase).
3346
- Various documentation issues, including `MemoryFS` docstring not rendering properly.
47+
- Avoid creating a new connection on every call of `FTPFS.upload`. Closes [#455](https://github.com/PyFilesystem/pyfilesystem2/issues/455).
48+
- `WrapReadOnly.removetree` not raising a `ResourceReadOnly` when called. Closes [#468](https://github.com/PyFilesystem/pyfilesystem2/issues/468).
49+
- `WrapCachedDir.isdir` and `WrapCachedDir.isfile` raising a `ResourceNotFound` error on non-existing path ([#470](https://github.com/PyFilesystem/pyfilesystem2/pull/470)).
50+
- `FTPFS` not listing certain entries with sticky/SUID/SGID permissions set by Linux server ([#473](https://github.com/PyFilesystem/pyfilesystem2/pull/473)).
51+
Closes [#451](https://github.com/PyFilesystem/pyfilesystem2/issues/451).
52+
- `scandir` iterator not being closed explicitly in `OSFS.scandir`, occasionally causing a `ResourceWarning`
53+
to be thrown. Closes [#311](https://github.com/PyFilesystem/pyfilesystem2/issues/311).
54+
- Incomplete type annotations for the `temp_fs` parameter of `WriteTarFS` and `WriteZipFS`.
55+
Closes [#410](https://github.com/PyFilesystem/pyfilesystem2/issues/410).
3456

3557

3658
## [2.4.12] - 2021-01-14

‎fs/_ftp_parse.py‎

Lines changed: 33 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@
1919
RE_LINUX = re.compile(
2020
r"""
2121
^
22-
([ldrwx-]{10})
22+
([-dlpscbD])
23+
([r-][w-][xsS-][r-][w-][xsS-][r-][w-][xtT-][\.\+]?)
2324
\s+?
2425
(\d+)
2526
\s+?
@@ -55,9 +56,7 @@
5556

5657

5758
def get_decoders():
58-
"""
59-
Returns all available FTP LIST line decoders with their matching regexes.
60-
"""
59+
"""Return all available FTP LIST line decoders with their matching regexes."""
6160
decoders = [
6261
(RE_LINUX, decode_linux),
6362
(RE_WINDOWSNT, decode_windowsnt),
@@ -110,14 +109,14 @@ def _decode_linux_time(mtime):
110109

111110

112111
def decode_linux(line, match):
113-
perms, links, uid, gid, size, mtime, name = match.groups()
114-
is_link = perms.startswith("l")
115-
is_dir = perms.startswith("d") or is_link
112+
ty, perms, links, uid, gid, size, mtime, name = match.groups()
113+
is_link = ty == "l"
114+
is_dir = ty == "d" or is_link
116115
if is_link:
117116
name, _, _link_name = name.partition("->")
118117
name = name.strip()
119118
_link_name = _link_name.strip()
120-
permissions = Permissions.parse(perms[1:])
119+
permissions = Permissions.parse(perms)
121120

122121
mtime_epoch = _decode_linux_time(mtime)
123122

@@ -148,13 +147,34 @@ def _decode_windowsnt_time(mtime):
148147

149148

150149
def decode_windowsnt(line, match):
151-
"""
152-
Decodes a Windows NT FTP LIST line like one of these:
150+
"""Decode a Windows NT FTP LIST line.
151+
152+
Examples:
153+
Decode a directory line::
154+
155+
>>> line = "11-02-18 02:12PM <DIR> images"
156+
>>> match = RE_WINDOWSNT.match(line)
157+
>>> pprint(decode_windowsnt(line, match))
158+
{'basic': {'is_dir': True, 'name': 'images'},
159+
'details': {'modified': 1518358320.0, 'type': 1},
160+
'ftp': {'ls': '11-02-18 02:12PM <DIR> images'}}
161+
162+
Decode a file line::
163+
164+
>>> line = "11-02-18 03:33PM 9276 logo.gif"
165+
>>> match = RE_WINDOWSNT.match(line)
166+
>>> pprint(decode_windowsnt(line, match))
167+
{'basic': {'is_dir': False, 'name': 'logo.gif'},
168+
'details': {'modified': 1518363180.0, 'size': 9276, 'type': 2},
169+
'ftp': {'ls': '11-02-18 03:33PM 9276 logo.gif'}}
170+
171+
Alternatively, the time might also be present in 24-hour format::
153172
154-
`11-02-18 02:12PM <DIR> images`
155-
`11-02-18 03:33PM 9276 logo.gif`
173+
>>> line = "11-02-18 15:33 9276 logo.gif"
174+
>>> match = RE_WINDOWSNT.match(line)
175+
>>> decode_windowsnt(line, match)["details"]["modified"]
176+
1518363180.0
156177
157-
Alternatively, the time (02:12PM) might also be present in 24-hour format (14:12).
158178
"""
159179
is_dir = match.group("size") == "<DIR>"
160180

‎fs/_repr.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ def make_repr(class_name, *args, **kwargs):
2727
>>> MyClass('Will')
2828
MyClass('foo', name='Will')
2929
>>> MyClass(None)
30-
MyClass()
30+
MyClass('foo')
3131
3232
"""
3333
arguments = [repr(arg) for arg in args]

‎fs/_url_tools.py‎

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,15 @@
1111

1212
def url_quote(path_snippet):
1313
# type: (Text) -> Text
14-
"""
15-
On Windows, it will separate drive letter and quote windows
16-
path alone. No magic on Unix-alie path, just pythonic
17-
`pathname2url`
14+
"""Quote a URL without quoting the Windows drive letter, if any.
15+
16+
On Windows, it will separate drive letter and quote Windows
17+
path alone. No magic on Unix-like path, just pythonic
18+
`~urllib.request.pathname2url`.
1819
1920
Arguments:
20-
path_snippet: a file path, relative or absolute.
21+
path_snippet (str): a file path, relative or absolute.
22+
2123
"""
2224
if _WINDOWS_PLATFORM and _has_drive_letter(path_snippet):
2325
drive_letter, path = path_snippet.split(":", 1)
@@ -34,17 +36,19 @@ def url_quote(path_snippet):
3436

3537
def _has_drive_letter(path_snippet):
3638
# type: (Text) -> bool
37-
"""
38-
The following path will get True
39-
D:/Data
40-
C:\\My Dcouments\\ test
39+
"""Check whether a path contains a drive letter.
4140
42-
And will get False
41+
Arguments:
42+
path_snippet (str): a file path, relative or absolute.
4343
44-
/tmp/abc:test
44+
Example:
45+
>>> _has_drive_letter("D:/Data")
46+
True
47+
>>> _has_drive_letter(r"C:\\System32\\ test")
48+
True
49+
>>> _has_drive_letter("/tmp/abc:test")
50+
False
4551
46-
Arguments:
47-
path_snippet: a file path, relative or absolute.
4852
"""
4953
windows_drive_pattern = ".:[/\\\\].*$"
5054
return re.match(windows_drive_pattern, path_snippet) is not None

0 commit comments

Comments
 (0)