🔥 Drop Python 2 support (#2105)

* 🔥 Remove `hostpython2` recipe

* 🔥 Remove `python2` recipe

* 🔥 Remove python2 related from `ci/constants`

* 🔥 Remove python2 environ/tests from `tox`

* 🔥 Remove python2 code from `PythonUtil.java`

* 🔥 Remove python2 patch from `recipes/numpy`

* 🔥 Remove python2 code from `ci/rebuild_updated_recipes`

* 🔥 Remove `test_entrypoints_python2.py`

* 🔥 Remove python2 code from `start.c`

* 🔥 Remove python2 testapps

* 🔥 Remove python2 code from `bs/common/build/build.py`

* 🍱 Update shebang: from `python2` to `python`

* 🍱 Update requirements for `testlauncher_setup`

* 🔥 Remove python2 from class `Bootstrap.recipe_depends`

* 🔥 Remove python2 code from `recipe.py`

* 🔥 Remove `Dockerfile.py2`

* 🍱 Update docstring example for `ToolchainCL.recipes`

`python2` it will not be a conflict anymore, since it doesn't exist.

* 🔥 Remove `note` about `python2` target

* 🍱 Update `android` recipe to avoid `python2`

* 🔥 Remove `python2`  from recipes's `depends`

* 🔥 Remove `hostpython2`  from recipes's `depends`

* 🔥 Remove `python2`  from recipes's `conflicts`

* 🍱 Change `hostpython2` to `hostpython3` for `mysqldb` recipe

* 🔥 Remove python2 code from `python` module

* ✅ Update tests to work without `python2`

* 🍱 Update `ndghttpsclient` recipe?

This is strange...that recipe should be in a folder right?

* 🔥 Remove python2 tests from `Makefile`, `travis.yml` and `push.yml`

* 🔥 Remove `make_python_zip`

Because we don't use it anymore. We used to package `python2` but
our `python3` was never zipped with that function, so there is no point
to maintain it now that we get rid of python2.

* 📝 Update docs

* Clean up README for Python 2 changes

* Update Python version support documentation

* Removed Python 2 app build from release process

This build was only there to check Python 2 support, so it's fine to skip it now.

* Made note about python2 support more explicit

* Changed a /usr/bin/env call to explicitly python3

I think being explicit is still best practice, correct me if I'm wrong.

We don't really support calling this script directly any more anyway, but I guess may as well do it right!

Co-authored-by: Alexander Taylor <alexanderjohntaylor@gmail.com>

Author: opacam

.github/workflows/push.yml

@@ -57,7 +57,7 @@ jobs:
     - name: Build apk for Python 3 ${{ matrix.build-arch }}
       run: |
         mkdir -p apks
-        make docker/run/make/with-artifact/testapps/python3/${{ matrix.build-arch }}
+        make docker/run/make/with-artifact/testapps/${{ matrix.build-arch }}
     - uses: actions/upload-artifact@v1
       with:
         name: bdisttest_python3_sqlite_openssl_googlendk__${{ matrix.build-arch }}-debug-1.1.apk

.travis.yml

@@ -36,9 +36,6 @@ jobs:
         - tox -- tests/ --ignore tests/test_pythonpackage.py
       name: "Tox Pep8"
       env: TOXENV=pep8
-    - <<: *unittests
-      name: "Tox Python 2"
-      env: TOXENV=py27
     - <<: *unittests
       name: "Tox Python 3 & Coverage"
       env: TOXENV=py3
@@ -48,7 +45,7 @@ jobs:
       name: Python 3 arm64-v8a (with numpy)
       stage: build testapps
       before_script: make docker/pull
-      script: make docker/run/make/testapps/python3/arm64-v8a
+      script: make docker/run/make/testapps/arm64-v8a
     - <<: *testapps
       name: Python 3 armeabi-v7a
       os: osx
@@ -57,10 +54,7 @@ jobs:
         # installs java 1.8, android's SDK/NDK and p4a
         - make -f ci/makefiles/osx.mk
         - export JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
-      script: make testapps/python3/armeabi-v7a PYTHON_WITH_VERSION=python3
-    - <<: *testapps
-      name: Python 2 armeabi-v7a
-      script: make docker/run/make/testapps/python2/armeabi-v7a
+      script: make testapps/armeabi-v7a PYTHON_WITH_VERSION=python3
     - <<: *testapps
       name: Rebuild updated recipes
       script: travis_wait 30 make docker/run/make/rebuild_updated_recipes

Dockerfile.py2

@@ -35,18 +35,13 @@ rebuild_updated_recipes: virtualenv
 	ANDROID_SDK_HOME=$(ANDROID_SDK_HOME) ANDROID_NDK_HOME=$(ANDROID_NDK_HOME) \
 	$(PYTHON) ci/rebuild_updated_recipes.py
 
-testapps/python2/armeabi-v7a: virtualenv
-	. $(ACTIVATE) && cd testapps/ && \
-    python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
-    --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools
-
-testapps/python3/arm64-v8a: virtualenv
+testapps/arm64-v8a: virtualenv
 	. $(ACTIVATE) && cd testapps/ && \
     python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
     --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools,numpy \
     --arch=arm64-v8a
 
-testapps/python3/armeabi-v7a: virtualenv
+testapps/armeabi-v7a: virtualenv
 	. $(ACTIVATE) && cd testapps/ && \
     python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
     --arch=armeabi-v7a
@@ -77,11 +72,7 @@ docker/run/make/%: docker/build
 	docker run --rm --env-file=.env $(DOCKER_IMAGE) make $*
 
 docker/run/make/with-artifact/%: docker/build
-ifeq (,$(findstring python3,$($*)))
 	$(eval $@_APP_NAME := bdisttest_python3_sqlite_openssl_googlendk)
-else
-	$(eval $@_APP_NAME := bdisttest_python2_sqlite_openssl)
-endif
 	$(eval $@_APP_ARCH := $(shell basename $*))
 	docker run --name p4a-latest --env-file=.env $(DOCKER_IMAGE) make $*
 	docker cp p4a-latest:/home/user/app/testapps/$($@_APP_NAME)__$($@_APP_ARCH)-debug-1.1-.apk ./apks

Makefile

@@ -102,23 +102,13 @@ new recipe for python3 (3.7.1) had a new build system which was
 applied to the ancient python recipe, allowing us to bump the python2
 version number to 2.7.15. This change unified the build process for
 both python recipes, and probably solved various issues detected over the
-years. It should also be mentioned that these **unified python recipes**
-require a **minimum target api level of 21**,
-*Android 5.0 - Lollipop*, so in the case that you need to build targeting an
+years. These **unified python recipes** require a **minimum target api level of 21**,
+*Android 5.0 - Lollipop*. If you need to build targeting an
 api level below 21, you should use an older version of python-for-android
 (<=0.7.1).
 
-Be aware that this project is in constant development so, as per time of writing,
-you should use a minimum on Android's NDK r19, and ``we recommend using NDK r19b``.
-This is because the toolchains installed by
-default with the NDK can be used *in-place* and the python-for-android project
-has been adapted for that feature. Also be aware that more recent versions of the
-Android's NDK may not work.
-
-Those mentioned changes has been done this way to make easier the transition
-between python3 and python2. We will slowly phase out python2 support
-towards 2020...so...if you are using python2 in your projects you should
-consider migrating it into python3.
+On March of 2020 we dropped support for creating apps that use Python 2. The latest 
+python-for-android release that supported building Python 2 was version 2019.10.6.
 
 ## Contributors
 

README.md

@@ -2,59 +2,12 @@
 
 
 class TargetPython(Enum):
-    python2 = 0
     python3 = 2
 
 
 # recipes that currently break the build
 # a recipe could be broken for a target Python and not for the other,
 # hence we're maintaining one list per Python target
-BROKEN_RECIPES_PYTHON2 = set([
-    # pythonhelpers.h:12:18: fatal error: string: No such file or directory
-    'atom',
-    # https://github.com/kivy/python-for-android/issues/550
-    'audiostream',
-    'brokenrecipe',
-    'evdev',
-    # distutils.errors.DistutilsError
-    # Could not find suitable distribution for Requirement.parse('cython')
-    'ffpyplayer',
-    'flask',
-    'groestlcoin_hash',
-    # https://github.com/kivy/python-for-android/issues/1354
-    'kiwisolver',
-    'libmysqlclient',
-    'libsecp256k1',
-    'libtribler',
-    'ndghttpsclient',
-    'm2crypto',
-    # ImportError: No module named setuptools
-    'netifaces',
-    'Pillow',
-    # depends on cffi that still seems to have compilation issues
-    'protobuf_cpp',
-    'xeddsa',
-    'x3dh',
-    'pynacl',
-    'doubleratchet',
-    'omemo',
-    # requires `libpq-dev` system dependency e.g. for `pg_config` binary
-    'psycopg2',
-    # most likely some setup in the Docker container, because it works in host
-    'pyjnius', 'pyopenal',
-    'pyproj',
-    'pysdl2',
-    'pyzmq',
-    'secp256k1',
-    'shapely',
-    # mpmath package with a version >= 0.19 required
-    'sympy',
-    'twisted',
-    'vlc',
-    'websocket-client',
-    'zeroconf',
-    'zope',
-])
 BROKEN_RECIPES_PYTHON3 = set([
     'brokenrecipe',
     # enum34 is not compatible with Python 3.6 standard library
@@ -76,11 +29,10 @@ class TargetPython(Enum):
 ])
 
 BROKEN_RECIPES = {
-    TargetPython.python2: BROKEN_RECIPES_PYTHON2,
     TargetPython.python3: BROKEN_RECIPES_PYTHON3,
 }
 # recipes that were already built will be skipped
 CORE_RECIPES = set([
     'pyjnius', 'kivy', 'openssl', 'requests', 'sqlite3', 'setuptools',
-    'numpy', 'android', 'hostpython2', 'hostpython3', 'python2', 'python3',
+    'numpy', 'android', 'hostpython3', 'python3',
 ])

ci/constants.py

@@ -19,15 +19,12 @@
   [ERROR]:   Didn't find any valid dependency graphs.
   [ERROR]:   This means that some of your requirements pull in conflicting dependencies.
 - only rebuilds on sdl2 bootstrap
-- supports mainly python3 with fallback to python2
 """
 import sh
 import os
 from pythonforandroid.build import Context
 from pythonforandroid import logger
-from pythonforandroid.graph import get_recipe_order_and_bootstrap
 from pythonforandroid.toolchain import current_directory
-from pythonforandroid.util import BuildInterruptingException
 from pythonforandroid.recipe import Recipe
 from ci.constants import TargetPython, CORE_RECIPES, BROKEN_RECIPES
 
@@ -55,11 +52,9 @@ def build(target_python, requirements):
     """
     if not requirements:
         return
-    testapp = 'setup_testapp_python2.py'
+    testapp = 'setup_testapp_python3_sqlite_openssl.py'
     android_sdk_home = os.environ['ANDROID_SDK_HOME']
     android_ndk_home = os.environ['ANDROID_NDK_HOME']
-    if target_python == TargetPython.python3:
-        testapp = 'setup_testapp_python3_sqlite_openssl.py'
     requirements.add(target_python.name)
     requirements = ','.join(requirements)
     logger.info('requirements: {}'.format(requirements))
@@ -91,16 +86,6 @@ def main():
                 'removed {} from recipes because deleted'.format(recipe_name)
             )
 
-    # forces the default target
-    recipes_and_target = recipes | set([target_python.name])
-    try:
-        build_order, python_modules, bs = get_recipe_order_and_bootstrap(
-            context, recipes_and_target, None)
-    except BuildInterruptingException:
-        # fallback to python2 if default target is not compatible
-        logger.info('incompatible with {}'.format(target_python.name))
-        target_python = TargetPython.python2
-        logger.info('falling back to {}'.format(target_python.name))
     # removing the known broken recipe for the given target
     broken_recipes = BROKEN_RECIPES[target_python]
     recipes -= broken_recipes

ci/rebuild_updated_recipes.py

@@ -8,38 +8,14 @@ This page contains instructions for using different build options.
 Python versions
 ---------------
 
-python2
-~~~~~~~
-
-Select this by adding it in your requirements, e.g. ``--requirements=python2``.
-
-This option builds Python 2.7.2 for your selected Android
-architecture. There are no special requirements, all the building is
-done locally.
-
-
-python3
-~~~~~~~
-
-Python3 is supported in two ways. The default method uses CPython 3.7+
-and works with any recent version of the Android NDK.
-
-Select Python 3 by adding it to your requirements,
-e.g. ``--requirements=python3``.
-
-.. note:: ctypes is not included automatically, if you would like to use it
-          then add libffi to your requirements,
-          e.g. ``--requirements=kivy,libffi,python3``.
-
-
-CrystaX python3
-~~~~~~~~~~~~~~~
+python-for-android supports using Python 3.7 or higher. To explicitly select a Python
+version in your requirements, use e.g. ``--requirements=python3==3.7.1,hostpython3==3.7.1``.
 
-python-for-android no longer supports building for Python 3 using the CrystaX
-NDK. Instead, use the python3 recipe, which can be built using the normal
-Google NDK.
+The last python-for-android version supporting Python2 was `v2019.10.06
+          <https://github.com/kivy/python-for-android/archive/v2019.10.06.zip>`__
 
-.. note:: The last python-for-android version supporting CrystaX was `0.7.0.
+Python-for-android no longer supports building for Python 3 using the CrystaX
+NDK. The last python-for-android version supporting CrystaX was `0.7.0.
           <https://github.com/kivy/python-for-android/archive/0.7.0.zip>`__
 
 .. _bootstrap_build_options:
@@ -60,7 +36,7 @@ sdl2
 ~~~~
 
 Use this with ``--bootstrap=sdl2``, or just include the
-``sdl2`` recipe, e.g. ``--requirements=sdl2,python2``.
+``sdl2`` recipe, e.g. ``--requirements=sdl2,python3``.
 
 SDL2 is a popular cross-platform depelopment library, particularly for
 games. It has its own Android project support, which
@@ -120,7 +96,7 @@ webview
 ~~~~~~~
 
 You can use this with ``--bootstrap=webview``, or include the
-``webviewjni`` recipe, e.g. ``--requirements=webviewjni,python2``.
+``webviewjni`` recipe, e.g. ``--requirements=webviewjni,python3``.
 
 The webview bootstrap gui is, per the name, a WebView displaying a
 webpage, but this page is hosted on the device via a Python

doc/source/buildoptions.rst

@@ -75,9 +75,6 @@ Release checklist
   - `python3 setup_testapp_python3_sqlite_openssl.py apk`
     - [ ] `armeabi-v7a`
     - [ ] `arm64-v8a`
-  - `python3 setup_testapp_python2.py apk`
-    - [ ] `armeabi-v7a`
-    - [ ] `arm64-v8a`
 - [ ] Check that the version number is correct