add challenge mirror, verify, format functionality (#134)

Adds `ctf challenge mirror <challenge>` and `ctf challenge verify <challenge>` adapted from #106 

Originally, this functionality was called `pull` and `verify` - however, `push` is already used to push challenge changes to the git repository. I think `mirror` is a better name, as ctfcli will attempt to mirror / copy the remote state from ctfd. This way  `pull` stays in its current git-like form, for git-related operations.

More additions:
- I've removed update / create / verify files - this can be achieved by just using --ignore=files.
- I've added `files_directory_name` (defaulting to `dist`) to specify where ctfcli should download the files, relative to challenge.yml
- I've added a warning when there are additional challenges on the remote, that are not registered locally
- `ctf challenge verify` will exit with status code 2 if the verification was successful, but some challenges are out of sync.
- I've fixed some typos

Thanks to @reteps for the initial contribution!

Closes: #101 #106

Author: Miłosz Skaza

README.md

@@ -20,48 +20,55 @@ Alternatively, you can always install it with `pip` as a python module:
 
 ## 1. Create an Event
 
-ctfcli turns the current folder into a CTF event git repo. It asks for the base url of the CTFd instance you're working with and an access token.
+Ctfcli turns the current folder into a CTF event git repo. 
+It asks for the base url of the CTFd instance you're working with and an access token.
 
 ```
 ❯ ctf init
 Please enter CTFd instance URL: https://demo.ctfd.io
 Please enter CTFd Admin Access Token: d41d8cd98f00b204e9800998ecf8427e
-Do you want to continue with https://demo.ctfd.io and d41d8cd98f00b204e9800998ecf8427e [y/N]: y
+Do you want to continue with https://demo.ctfd.io and d41d8cd98f00b204e9800998ecf8427e [Y/n]: y
 Initialized empty Git repository in /Users/user/Downloads/event/.git/
 ```
 
-This will create the `.ctf` folder with the `config` file that will specify the URL, access token, and keep a record of all the challenges dedicated for this event.
+This will create the `.ctf` folder with the `config` file that will specify the URL, access token, and keep a record of
+all the challenges dedicated for this event.
 
 ## 2. Add challenges
 
-Events are made up of challenges. Challenges can be made from a subdirectory or pulled from another repository. Remote challenges are pulled into the event repo and a reference is kept in the `.ctf/config` file.
+Events are made up of challenges.
+Challenges can be made from a subdirectory or pulled from another repository.
+GIT-enabled challenges are pulled into the event repo, and a reference is kept in the `.ctf/config` file.
 
 ```
 ❯ ctf challenge add [REPO | FOLDER]
 ```
 
+##### Local folder:
 ```
 ❯ ctf challenge add crypto/stuff
 ```
 
+##### GIT repository:
 ```
 ❯ ctf challenge add https://github.com/challenge.git
-challenge
 Cloning into 'challenge'...
-remote: Enumerating objects: 624, done.
-remote: Counting objects: 100% (624/624), done.
-remote: Compressing objects: 100% (540/540), done.
-remote: Total 624 (delta 109), reused 335 (delta 45), pack-reused 0
-Receiving objects: 100% (624/624), 6.49 MiB | 21.31 MiB/s, done.
-Resolving deltas: 100% (109/109), done.
+[...]
+```
+
+##### GIT repository to a specific subfolder:
+```
+❯ ctf challenge add https://github.com/challenge.git crypto
+Cloning into 'crypto/challenge'...
+[...]
 ```
 
 ## 3. Install challenges
 
-Installing a challenge will automatically create the challenge in your CTFd instance using the API.
+Installing a challenge will create the challenge in your CTFd instance using the API.
 
 ```
-❯ ctf challenge install [challenge.yml | DIRECTORY]
+❯ ctf challenge install [challenge]
 ```
 
 ```
@@ -72,12 +79,13 @@ Installing buffer_overflow
 Success!
 ```
 
-## 4. Update challenges
+## 4. Sync challenges
 
-Syncing a challenge will automatically update the challenge in your CTFd instance using the API. Any changes made in the `challenge.yml` file will be reflected in your instance.
+Syncing a challenge will update the challenge in your CTFd instance using the API. 
+Any changes made in the `challenge.yml` file will be reflected in your instance.
 
 ```
-❯ ctf challenge sync [challenge.yml | DIRECTORY]
+❯ ctf challenge sync [challenge]
 ```
 
 ```
@@ -88,6 +96,70 @@ Syncing buffer_overflow
 Success!
 ```
 
+## 5. Deploy services
+
+Deploying a challenge will automatically create the challenge service (by default in your CTFd instance).
+You can also use a different deployment handler to deploy the service via SSH to your own server, 
+or a separate docker registry.
+
+The challenge will also be automatically installed or synced.
+Obtained connection info will be added to your `challenge.yml` file.
+```
+❯ ctf challenge deploy [challenge]
+```
+
+```
+❯ ctf challenge deploy web-1
+Deploying challenge service 'web-1' (web-1/challenge.yml) with CloudDeploymentHandler ...
+Challenge service deployed at: https://web-1-example-instance.chals.io
+Updating challenge 'web-1'
+Success!
+```
+
+## 6. Verify challenges
+
+Verifying a challenge will check if the local version of the challenge is the same as one installed in your CTFd instance.
+
+```
+❯ ctf challenge verify [challenge]
+```
+
+```
+❯ ctf challenge verify buffer_overflow
+Verifying challenges  [------------------------------------]    0%
+Verifying challenges  [####################################]  100%
+Success! All challenges verified!
+Challenges in sync:
+ - buffer_overflow
+```
+
+## 7. Mirror changes
+
+Mirroring a challenge is the reverse operation to syncing.
+It will update the local version of the challenge with details of the one installed in your CTFd instance.
+It will also issue a warning if you have any remote challenges that are not tracked locally.
+
+```
+❯ ctf challenge mirror [challenge]
+```
+
+```
+❯ ctf challenge verify buffer_overflow
+Mirorring challenges  [------------------------------------]    0%
+Mirorring challenges  [####################################]  100%
+Success! All challenges mirrored!
+```
+
+## Operations on all challenges
+
+You can perform operations on all challenges defined in your config by simply skipping the challenge parameter.
+
+- `ctf challenge install`
+- `ctf challenge sync`
+- `ctf challenge deploy`
+- `ctf challenge verify`
+- `ctf challenge mirror`
+
 # Challenge Templates
 
 `ctfcli` contains pre-made challenge templates to make it faster to create CTF challenges with safe defaults.
@@ -126,6 +198,6 @@ The specification format has already been tested and used with CTFd in productio
 
 # Plugins
 
-`ctfcli` plugins are essentially additions to to the command line interface via dynamic class modifications. See the [plugin documentation page](docs/plugins.md) for a simple example.
+`ctfcli` plugins are essentially additions to the command line interface via dynamic class modifications. See the [plugin documentation page](docs/plugins.md) for a simple example.
 
 *`ctfcli` is an alpha project! The plugin interface is likely to change!*

ctfcli/cli/challenges.py

@@ -693,7 +693,7 @@ def deploy(
                 elif deployment_result.connection_info:
                     challenge["connection_info"] = deployment_result.connection_info
 
-                # Finally if no connection_info was provided in the challenge and the
+                # Finally, if no connection_info was provided in the challenge and the
                 # deployment didn't result in one either, just ensure it's not present
                 else:
                     challenge["connection_info"] = None
@@ -714,6 +714,8 @@ def deploy(
                         f"Challenge service deployed at: {challenge['connection_info']}",
                         fg="green",
                     )
+
+                    challenge.save()  # Save the challenge with the new connection_info
                 else:
                     click.secho(
                         "Could not resolve a connection_info for the deployed service.\nIf your DeploymentHandler "
@@ -793,8 +795,8 @@ def lint(
         click.secho("Success! Lint didn't find any issues!", fg="green")
         return 0
 
-    def healthcheck(self, challenge: str = None):
-        log.debug(f"lint: (challenge={challenge})")
+    def healthcheck(self, challenge: str = None) -> int:
+        log.debug(f"healthcheck: (challenge={challenge})")
         config = Config()
         challenge_path = Path.cwd()
 
@@ -861,3 +863,193 @@ def healthcheck(self, challenge: str = None):
 
         click.secho("Success! Challenge passed the healthcheck.", fg="green")
         return 0
+
+    def mirror(
+        self,
+        challenge: str = None,
+        files_directory: str = "dist",
+        skip_verify: bool = False,
+        ignore: Union[str, Tuple[str]] = (),
+    ) -> int:
+        config = Config()
+        challenge_keys = [challenge]
+
+        # Get all local challenges if not specifying a challenge
+        if challenge is None:
+            challenge_keys = config.challenges.keys()
+
+        # Check if there are attributes to be ignored, and if there's only one cast it to a tuple
+        if isinstance(ignore, str):
+            ignore = (ignore,)
+
+        # Load local challenges
+        local_challenges, failed_mirrors = [], []
+        for challenge_key in challenge_keys:
+            challenge_path = config.project_path / Path(challenge_key)
+
+            if not challenge_path.name.endswith(".yml"):
+                challenge_path = challenge_path / "challenge.yml"
+
+            try:
+                local_challenges.append(Challenge(challenge_path))
+
+            except ChallengeException as e:
+                click.secho(str(e), fg="red")
+                failed_mirrors.append(challenge_key)
+                continue
+
+        remote_challenges = Challenge.load_installed_challenges()
+
+        if len(challenge_keys) > 1:
+            # When mirroring all challenges - issue a warning if there are extra challenges on the remote
+            # that do not have a local version
+            local_challenge_names = [c["name"] for c in local_challenges]
+
+            for remote_challenge in remote_challenges:
+                if remote_challenge["name"] not in local_challenge_names:
+                    click.secho(
+                        f"Found challenge '{remote_challenge['name']}' in CTFd, but not in .ctf/config\n"
+                        "Mirroring does not create new local challenges\n"
+                        "Please add the local challenge if you wish to manage it with ctfcli\n",
+                        fg="yellow",
+                    )
+
+        with click.progressbar(local_challenges, label="Mirroring challenges") as challenges:
+            for challenge in challenges:
+                try:
+                    if not skip_verify and challenge.verify(ignore=ignore):
+                        click.secho(
+                            f"Challenge '{challenge['name']}' is already in sync. Skipping mirroring.",
+                            fg="blue",
+                        )
+                    else:
+                        # if skip_verify is True or challenge.verify(ignore=ignore) is False
+                        challenge.mirror(files_directory_name=files_directory, ignore=ignore)
+
+                except ChallengeException as e:
+                    click.secho(str(e), fg="red")
+                    failed_mirrors.append(challenge["name"])
+
+        if len(failed_mirrors) == 0:
+            click.secho("Success! All challenges mirrored!", fg="green")
+            return 0
+
+        click.secho("Mirror failed for:", fg="red")
+        for challenge in failed_mirrors:
+            click.echo(f" - {challenge}")
+
+        return 1
+
+    def verify(self, challenge: str = None, ignore: Tuple[str] = ()) -> int:
+        config = Config()
+        challenge_keys = [challenge]
+
+        # Get all local challenges if not specifying a challenge
+        if challenge is None:
+            challenge_keys = config.challenges.keys()
+
+        # Check if there are attributes to be ignored, and if there's only one cast it to a tuple
+        if isinstance(ignore, str):
+            ignore = (ignore,)
+
+        # Load local challenges
+        local_challenges, failed_verifications = [], []
+        for challenge_key in challenge_keys:
+            challenge_path = config.project_path / Path(challenge_key)
+
+            if not challenge_path.name.endswith(".yml"):
+                challenge_path = challenge_path / "challenge.yml"
+
+            try:
+                local_challenges.append(Challenge(challenge_path))
+
+            except ChallengeException as e:
+                click.secho(str(e), fg="red")
+                failed_verifications.append(challenge_key)
+                continue
+
+        remote_challenges = Challenge.load_installed_challenges()
+
+        if len(challenge_keys) > 1:
+            # When verifying all challenges - issue a warning if there are extra challenges on the remote
+            # that do not have a local version
+            local_challenge_names = [c["name"] for c in local_challenges]
+
+            for remote_challenge in remote_challenges:
+                if remote_challenge["name"] not in local_challenge_names:
+                    click.secho(
+                        f"Found challenge '{remote_challenge['name']}' in CTFd, but not in .ctf/config\n"
+                        "Please add the local challenge if you wish to manage it with ctfcli\n",
+                        fg="yellow",
+                    )
+
+        challenges_in_sync, challenges_out_of_sync = [], []
+        with click.progressbar(local_challenges, label="Verifying challenges") as challenges:
+            for challenge in challenges:
+                try:
+                    if not challenge.verify(ignore=ignore):
+                        challenges_out_of_sync.append(challenge["name"])
+                    else:
+                        challenges_in_sync.append(challenge["name"])
+
+                except ChallengeException as e:
+                    click.secho(str(e), fg="red")
+                    failed_verifications.append(challenge["name"])
+
+        if len(failed_verifications) == 0:
+            click.secho("Success! All challenges verified!", fg="green")
+
+            if len(challenges_in_sync) > 0:
+                click.secho("Challenges in sync:", fg="green")
+                for challenge in challenges_in_sync:
+                    click.echo(f" - {challenge}")
+
+            if len(challenges_out_of_sync) > 0:
+                click.secho("Challenges out of sync:", fg="yellow")
+                for challenge in challenges_out_of_sync:
+                    click.echo(f" - {challenge}")
+
+            if len(challenges_out_of_sync) > 1:
+                return 2
+
+            return 1
+
+        click.secho("Verification failed for:", fg="red")
+        for challenge in failed_verifications:
+            click.echo(f" - {challenge}")
+
+        return 1
+
+    def format(self, challenge: str = None) -> int:
+        config = Config()
+        challenge_keys = [challenge]
+
+        # Get all local challenges if not specifying a challenge
+        if challenge is None:
+            challenge_keys = config.challenges.keys()
+
+        failed_formats = []
+        for challenge_key in challenge_keys:
+            challenge_path = config.project_path / Path(challenge_key)
+
+            if not challenge_path.name.endswith(".yml"):
+                challenge_path = challenge_path / "challenge.yml"
+
+            try:
+                # load the challenge and save it without changes
+                Challenge(challenge_path).save()
+
+            except ChallengeException as e:
+                click.secho(str(e), fg="red")
+                failed_formats.append(challenge_key)
+                continue
+
+        if len(failed_formats) == 0:
+            click.secho("Success! All challenges formatted!", fg="green")
+            return 0
+
+        click.secho("Format failed for:", fg="red")
+        for challenge in failed_formats:
+            click.echo(f" - {challenge}")
+
+        return 1