diff --git a/docs/API.rst b/docs/API.rst index cc307581ae..8e85be0645 100644 --- a/docs/API.rst +++ b/docs/API.rst @@ -15,41 +15,49 @@ All the log-in methods and restrictions that apply to normal ContestWebServer usage also apply for API usage. In particular, this implies that IP autologin, if configured, can be used for API calls. -Otherwise, making a `POST` request to the `/api/login` endpoint, passing -`username` and `password` as form data (as is done for regular login), will +Otherwise, making a ``POST`` request to the ``/api/login`` endpoint, passing +``username`` and ``password`` as form data (as is done for regular login), will return a JSON object with the following structure: .. sourcecode:: json - {"login_data": "string"} + {"login_data": /* string */ } The content of the returned login data string should be passed to API calls -that require authentication in a `X-CMS-Authorization` header. +that require authentication in a ``X-CMS-Authorization`` header. All other API methods described here require authentication. Task list ========= -An authenticated `GET` request to `/api/task_list`, done while the contest is -running, will return the following object: +An authenticated ``GET`` request to ``/api/task_list``, done while the contest +is running, will return the following object: .. sourcecode:: json - {"tasks": [{"name": "string", "statements": ["string"], "submission_format": ["string"] }]} + { + "tasks": [ + { + "name": /* string */, + "statements": [/* strings */], + "submission_format": [/* strings */] + } + ] + } -Tasks are ordered in the same order as in the UI. The `name` of the task is -used for additional task-specific API calls. `statements` contains a list of -languages for which a statement is available. `submission_format` contains a +Tasks are ordered in the same order as in the UI. The ``name`` of the task is +used for additional task-specific API calls. ``statements`` contains a list of +languages for which a statement is available. ``submission_format`` contains a list of files that will need to be submitted, where filenames containing the -`%l` string represent source files (that will be compiled and run on the +``%l`` string represent source files (that will be compiled and run on the server), and all other filenames represent output files. Submit ====== -An authenticated `POST` request to `/api/{taskname}/submit` will send a +An authenticated ``POST`` request to ``/api/{taskname}/submit`` will send a submission. Files should be provided (according to the submission format) as form data, with field names matching the submission format. @@ -57,36 +65,41 @@ The request will return an object with the ID of the new submission: .. sourcecode:: json - {"id": "string"} + {"id": /* string */} List submissions ================ -An authenticated `GET` request to `/api/{taskname}/submission_list` will return -an object describing all the submission done so far on the given task, in -chronological order: +An authenticated ``GET`` request to ``/api/{taskname}/submission_list`` will +return an object describing all the submission done so far on the given task, +in chronological order: .. sourcecode:: json - {"list": [{"id": "string"}]} + { + "list": [ + { "id": /* string */ } + ] + } Task statement ============== A PDF version of the task statement for a given language can be retrieved by -making an authenticated `GET` request to `tasks/{taskname}/statements/{lang}`. +making an authenticated ``GET`` request to +``tasks/{taskname}/statements/{lang}``. Scoring information =================== -An authenticated `GET` request to `/tasks/{taskname}/submissions/{id}` will -retrieve information on the submission with the given `id`. In particular, -its field `public_score` will contain the score of this submission, and -`task_public_score` will contain the current score for the task. +An authenticated ``GET`` request to ``/tasks/{taskname}/submissions/{id}`` will +retrieve information on the submission with the given ``id``. In particular, +its field ``public_score`` will contain the score of this submission, and +``task_public_score`` will contain the current score for the task. Additional details on the submission's results can be retrieved by making an -authenticated `GET` request to `/tasks/{taskname}/submissions/{id}/details`. +authenticated ``GET`` request to ``/tasks/{taskname}/submissions/{id}/details``. The endpoint will return an HTML snippet matching what is seen by contestants. Impersonation of users @@ -94,29 +107,31 @@ Impersonation of users Administrators may impersonate a user and perform requests on their behalf. -This is accomplished by using the authentication endpoint without `password` -and with an additional `admin_token` parameter equal to the `contest_admin_token` -from the CMS configuration. The returned authentication token refers to the -given user, but it is marked as impersonated. +This is accomplished by using the authentication endpoint without ``password`` +and with an additional ``admin_token`` parameter equal to the +``contest_admin_token`` from the CMS configuration. The returned authentication +token refers to the given user, but it is marked as impersonated. -Requests carrying an impersonated authentication token may bypass certain restrictions: +Requests carrying an impersonated authentication token may bypass certain +restrictions: * IP-based login restrictions do no apply. (But if IP-based autologin is set, it overrides all authentication tokens including impersonated ones.) * Hidden participation is never blocked. -* Requests can carry special parameters (either in the URL or in `POST` data) +* Requests can carry special parameters (either in the URL or in ``POST`` data) that bypasses further restrictions when set to ``1``: - * `override_phase_check` lets the operation proceed regardless of contest phase - (for example, you can submit even though the contest has already ended). + * ``override_phase_check`` lets the operation proceed regardless of contest + phase (for example, you can submit even though the contest has already + ended). - * `override_official` (in the submit endpoint) makes the submission count as - official regardless of contest phase. + * ``override_official`` (in the submit endpoint) makes the submission count + as official regardless of contest phase. - * `override_max_number` (in the submit endpoint) skips all checks for the + * ``override_max_number`` (in the submit endpoint) skips all checks for the maximum number of submits. - * `override_min_interval` (in the submit endpoint) skips all checks for the - minimum time interval between submits. + * ``override_min_interval`` (in the submit endpoint) skips all checks for + the minimum time interval between submits. diff --git a/docs/Docker image.rst b/docs/Docker image.rst index dfbffa8b5e..7099f9acd6 100644 --- a/docs/Docker image.rst +++ b/docs/Docker image.rst @@ -9,7 +9,7 @@ dependencies. We also provide: spawn a volatile database (not persisted on disk) as well as a CMS instance that automatically runs all unit tests and functional tests. -* :gh_blob:`docker/cms-stresstest.sh`: Similar to `cms_test.sh` but runs the stress +* :gh_blob:`docker/cms-stresstest.sh`: Similar to ``cms_test.sh`` but runs the stress tests instead of the unit tests. The stress test consists of: creating a database, populating it with some sample tasks, and then simulating some users logging in via ContestWebServer and repeatedly performing actions @@ -17,7 +17,7 @@ dependencies. We also provide: * :gh_blob:`docker/cms-dev.sh`: This file uses :gh_blob:`docker/docker-compose.dev.yml` to spawn a database (**persisted** in the local ``.dev/postgres-data`` folder - within the repository) as well as a CMS container that only runs `bash`, + within the repository) as well as a CMS container that only runs ``bash``, leaving you with a shell from where you can start cms services. Changes made in the repository are also reflected directly inside the container (the source code is mounted as a docker volume). The DB port and CMS server @@ -60,7 +60,7 @@ is similar to: script uses the **name of the current git branch** and passes it to ``-p``. Note also that if you are not part of the ``docker`` group then you'll need - to run every docker command with ``sudo``, including ``sudo docker//cms-test.sh``. + to run every docker command with ``sudo``, including ``sudo docker/cms-test.sh``. We recommend adding yourself to the ``docker`` group. What the ``docker/cms-test.sh`` command does is: first build a fresh CMS image when diff --git a/docs/External contest formats.rst b/docs/External contest formats.rst index 620c4dd7ad..429f51fd2b 100644 --- a/docs/External contest formats.rst +++ b/docs/External contest formats.rst @@ -19,7 +19,7 @@ As for the second set of needs, the philosophy is that CMS should not force upon At the moment, CMS comes with two loaders pre-installed: * :file:`italy_yaml`, for tasks/users stored in the "Italian Olympiad" format. -* :file:`polygon_xml`, for tasks made with `Polygon `_. +* :file:`polygon_xml`, for tasks made with `Polygon `__. The first one is not particularly suited for general use (see below for more details), so, if you don't want to migrate to one of the aforementioned formats then we encourage you to **write a loader** for your favorite format and then get in touch with CMS authors to have it accepted in CMS. See the file :gh_blob:`cmscontrib/loaders/base_loader.py` for some hints. @@ -171,7 +171,7 @@ The following are optional keys that must be present for some task type or score Polygon format ============== -`Polygon `_ is a popular platform for the creation of tasks, and a task format, used among others by Codeforces. +`Polygon `__ is a popular platform for the creation of tasks, and a task format, used among others by Codeforces. Since Polygon doesn't support CMS directly, some task parameters cannot be set using the standard Polygon configuration. The importer reads from an optional file :file:`cms_conf.py` additional configuration specifics to CMS. Additionally, user can add file named contestants.txt to allow importing some set of users. diff --git a/docs/Installation.rst b/docs/Installation.rst index e3a26f94b8..e5eba765db 100644 --- a/docs/Installation.rst +++ b/docs/Installation.rst @@ -81,7 +81,7 @@ On Ubuntu 24.04, one will need to run the following script as root to satisfy al apt install nginx-full php-cli texlive-latex-base \ a2ps ghc rustc mono-mcs pypy3 -The above commands provide a very essential Pascal environment. Consider installing the following packages for additional units: `fp-units-base`, `fp-units-fcl`, `fp-units-misc`, `fp-units-math` and `fp-units-rtl`. +The above commands provide a very essential Pascal environment. Consider installing the following packages for additional units: ``fp-units-base``, ``fp-units-fcl``, ``fp-units-misc``, ``fp-units-math`` and ``fp-units-rtl``. Arch Linux ---------- @@ -130,10 +130,10 @@ First obtain the source code of CMS. Download :gh_download:`CMS release` Alternatively, if you like living at the bleeding edge, check out the CMS `Git repository `_ instead. -The preferred method of installation is using :samp:`./install.py --dir={target} cms`, +The preferred method of installation is using :samp:`./install.py --dir={install_dir} cms`, which does the following: -* Creates a *target directory* of the given name. It contains a Python +* Creates an *installation directory* of the given name. It contains a Python virtual environment and subdirectories where CMS stores its data, logs, and caches. If you omit the ``--dir`` option, CMS is installed to ``~/cms`` (``cms`` in the home directory of the current user). Make sure that it is different from the @@ -143,25 +143,25 @@ which does the following: * Checks that Isolate is available. -* Installs the sample configuration files to :samp:`{target}/etc/cms.toml` - and :samp:`{target}/etc/cms_ranking.toml`. +* Installs the sample configuration files to :samp:`{install_dir}/etc/cms.toml` + and :samp:`{install_dir}/etc/cms_ranking.toml`. -Now you can run CMS commands from the shell directly as :samp:`{target}/bin/{command}`. +Now you can run CMS commands from the shell directly as :samp:`{install_dir}/bin/{command}`. It is usually more convenient to activate the virtual environment, which adds -:samp:`{target}/bin` to your ``$PATH``. This can be done by adding the following line +:samp:`{install_dir}/bin` to your ``$PATH``. This can be done by adding the following line to your ``~/.profile``: .. sourcecode:: bash source $TARGET/bin/activate -(with ``$TARGET`` replaced by the path to your target directory). +(with ``$TARGET`` replaced by the path to your installation directory). Development installs -------------------- -If you want to develop CMS, you can use :samp:`./install.py --dir={target} cms --devel --editable`. +If you want to develop CMS, you can use :samp:`./install.py --dir={install_dir} cms --devel --editable`. This includes development dependencies. It also makes the installation linked to the source directory, so you don't need to reinstall if you edit the source. @@ -180,6 +180,8 @@ and running the ``isolate-check-environment`` command which checks your system for common issues. +.. _installation_updatingcms: + Updating CMS ============ diff --git a/docs/Internals.rst b/docs/Internals.rst index 93143ad87c..8fbd3ac1c8 100644 --- a/docs/Internals.rst +++ b/docs/Internals.rst @@ -27,16 +27,18 @@ lines inside string represented in the JSON have to be escaped anyway). An RPC request must be of the form (it is pretty printed here, but it -is sent in compact form inside CMS):: +is sent in compact form inside CMS): - { - "__method": , - "__data": { - : , - ... - }, - "__id": - } +.. sourcecode:: json + + { + "__method": /* name of the requested method */, + "__data": { + /* name of first arg */: /* value of first arg */, + /* ... */ + }, + "__id": /* random ID string */ + } The arguments in ``__data`` are (of course) not ordered: they have to be matched according to their names. In particular, this means that @@ -47,13 +49,15 @@ The ``__id`` is a random string that will be returned in the response, and it is useful (actually, it's the only way) to match requests with responses. -The response is of the form:: +The response is of the form: + +.. sourcecode:: json - { - "__data": , - "__error": , - "__id": - } + { + "__data": /* return value or null */, + "__error": /* null or error string */, + "__id": /* random ID string */ + } The value of ``__id`` must of course be the same as in the request. If ``__error`` is not null, then ``__data`` is expected to be null. @@ -65,7 +69,7 @@ Setting the ``backdoor`` configuration key to true causes services to serve a Python console (accessible with netcat), running in the same interpreter instance as the service, allowing to inspect and modify its data, live. It will be bound to a local UNIX domain socket, usually at -:file:`{target}/run/{service}_{shard}`. +:file:`{install_dir}/run/{service}_{shard}`. Although there's no authentication mechanism to prevent unauthorized access, the socket is accessible only to the ``cmsuser``, which should make it safe to run the backdoor everywhere, even on workers that are diff --git a/docs/RankingWebServer.rst b/docs/RankingWebServer.rst index 99548bc84f..47f89677eb 100644 --- a/docs/RankingWebServer.rst +++ b/docs/RankingWebServer.rst @@ -52,7 +52,7 @@ RWS doesn't use the PostgreSQL database. Instead, it stores its data in :file:`/ The intended way to get data to RWS is to have the rest of CMS send it. The service responsible for that is ProxyService (PS for short). When PS is started for a certain contest, it will send the data for that contest to all RWSs it knows about (i.e. those in its configuration). This data includes the contest itself (its name, its begin and end times, etc.), its tasks, its users and teams, and the submissions received so far. Then it will continue to send new submissions as soon as they are scored and it will update them as needed (for example when a user uses a token). Note that hidden users (and their submissions) will not be sent to RWS. -There are also other ways to insert data into RWS: send custom HTTP requests or directly write JSON files. For the former, the script `cmsRWSHelper` can be used to handle the low level communication. +There are also other ways to insert data into RWS: send custom HTTP requests or directly write JSON files. For the former, the script ``cmsRWSHelper`` can be used to handle the low level communication. Logo, flags and faces --------------------- diff --git a/docs/Running CMS.rst b/docs/Running CMS.rst index 4845d682c0..9138ca4b53 100644 --- a/docs/Running CMS.rst +++ b/docs/Running CMS.rst @@ -51,7 +51,7 @@ Finally you have to create the database schema for CMS, by running: Configuring CMS =============== -There are two configuration files, one for CMS itself and one for the rankings. Samples for both files are in the directory :gh_tree:`config/`. You want to copy them to the same file names but without the ``.sample`` suffix (that is, to :file:`config/cms.toml` and :file:`config/cms_ranking.toml`) before modifying them. +There are two configuration files, one for CMS itself and one for the rankings. Samples for both files are in the directory :gh_tree:`config/`. To modify them, first install CMS (using, for example, ``./install.py cms``) and then modify the files in :samp:`{install_dir}/etc/`. * :file:`cms.toml` is intended to be the same on all machines; all configurations options are explained in the file; of particular importance is the definition of ``core_services``, that specifies where and how many services are going to be run, and the connecting line for the database, in which you need to specify the name of the user created above and its password. @@ -69,12 +69,10 @@ If you are organizing a real contest, you must also change ``secret_key`` to a r As the name implies, the value of ``secret_key`` must be kept confidential. If a contestant knows it (for example because you are using the default value), they may be easily able to log in as another contestant. -The configuration files get copied automatically by the ``prerequisites.py`` script, so you can either run ``sudo ./prerequisites.py install`` again (answering "Y" when questioned about overwriting old configuration files) or you could simply edit the previously installed configuration files (which are usually found in ``/usr/local/etc/`` or ``/etc/``), if you do not plan on running that command ever again. - Running CMS =========== -Here we will assume you installed CMS. If not, you should replace all commands path with the appropriate local versions (for example, ``cmsLogService`` becomes :gh_blob:`./scripts/cmsLogService`). +You must install CMS (using, for example, ``./install.py cms``) before continuing. Note that if you didn't activate CMS's virtual environment, you need to prefix all commands with :samp:`{install_dir}/bin/`. At this point, you should have CMS installed on all the machines you want run services on, with the same configuration file, and a running PostgreSQL instance. To run CMS, you need a contest in the database. To create a contest, follow :doc:`these instructions `. @@ -106,7 +104,7 @@ You should now be able to start exploring the admin interface, by default at htt cmsAddAdmin name -CMS will create an admin account with username "name" and a random password that will be printed by the command. You can log in with this credentials, and then use the admin interface to modify the account or add other accounts. +CMS will create an admin account with username "name" and a random password that will be printed by the command. You can log in with these credentials, and then use the admin interface to modify the account or add other accounts. .. _running-cms_recommended-setup: @@ -117,8 +115,8 @@ Starting CMS by systemd If your system runs ``systemd``, you can start parts of CMS as systemd services. They are usually managed by the user instance of systemd for the ``cmsuser`` account. -:samp:`./install.py --dir={target} systemd` installs the following services under -``~/.config/systemd/user/``: +:samp:`./install.py --dir={install_dir} systemd` installs the following +services under ``~/.config/systemd/user/``: * ``cms-logging.service`` that starts the ``cmsLogService``. diff --git a/docs/Score types.rst b/docs/Score types.rst index b0204858df..384011198d 100644 --- a/docs/Score types.rst +++ b/docs/Score types.rst @@ -62,4 +62,4 @@ For Batch tasks, this means that the tasks creates the outcome through a compara Custom score types ================== -Additional score types can be defined if necessary. This works in the same way :ref:`as with task types `: the classes need to extend :py:class:`cms.grading.scoretypes.ScoreType` and the entry point group is called `cms.grading.scoretypes`. +Additional score types can be defined if necessary. This works in the same way :ref:`as with task types `: the classes need to extend :py:class:`cms.grading.scoretypes.ScoreType` and the entry point group is called ``cms.grading.scoretypes``. diff --git a/docs/Task types.rst b/docs/Task types.rst index 6bded206fb..ccb30a5700 100644 --- a/docs/Task types.rst +++ b/docs/Task types.rst @@ -83,7 +83,7 @@ In practice, Communication tasks have two processes, running in two different sa This setup ensures that the contestant's code cannot access forbidden data, even in the case they have full knowledge of the admin code. -The admins must provide an executable manager called ``manager``. It can read the testcase input from stdin, and will also receive as argument the filenames of two FIFOs, from and to the contestant process (in this order). It must write to stdout the outcome and to stderr the message for the contestant (see :ref:`details about the format``). If the contestant's process fails, the output of the manager is ignored, and the outcome will be 0.0 and the message will explain the reason. +The admins must provide an executable manager called ``manager``. It can read the testcase input from stdin, and will also receive as argument the filenames of two FIFOs, from and to the contestant process (in this order). It must write to stdout the outcome and to stderr the message for the contestant (see :ref:`details about the format`). If the contestant's process fails, the output of the manager is ignored, and the outcome will be 0.0 and the message will explain the reason. Admins can also provide a manager called :file:`stub.{ext}` for each allowed language, where :file:`{ext}` is the standard extension of a source file in that language. The task type can be set up to compile the stub with the contestant's source. Usually, a stub takes care of the communication with the manager, so that the contestants have to implement only a function. As for Batch, admins can also add header file that will be used when compiling the stub and the contestant's source. @@ -198,4 +198,4 @@ Once that is done, install the distribution by executing CMS needs to be restarted for it to pick up the new task type. -For additional information see the `general distutils documentation `_ and the `section of the setuptools documentation about entry points `_. +For additional information see the `setuptools user guide `_ and specifically the `section about entry points `_. diff --git a/docs/Troubleshooting.rst b/docs/Troubleshooting.rst index 712a2cdaf5..1d00feebe4 100644 --- a/docs/Troubleshooting.rst +++ b/docs/Troubleshooting.rst @@ -18,7 +18,7 @@ Database - *Symptom.* Some components of CMS fail randomly and PostgreSQL complains about having too many connections. - *Possible cause.* The default configuration of PostgreSQL may allow insufficiently many incoming connections on the database engine. You can raise this limit by tweaking the ```max_connections``` parameter in ```postgresql.conf``` (`see docs `_). This, in turn, requires more shared memory for the PostgreSQL process (see ```shared_buffers``` parameter in `docs `_), which may overflow the maximum limit allowed by the operating system. In such case see the suggestions in http://www.postgresql.org/docs/9.1/static/kernel-resources.html#SYSVIPC. Users reported that another way to go is to use a connection pooler like `PgBouncer `_. + *Possible cause.* The default configuration of PostgreSQL may allow insufficiently many incoming connections on the database engine. You can raise this limit by tweaking the ``max_connections`` parameter in ``postgresql.conf`` (`see docs `_). This, in turn, requires more shared memory for the PostgreSQL process (see ``shared_buffers`` parameter in `docs `_), which may overflow the maximum limit allowed by the operating system. In such case see the suggestions in http://www.postgresql.org/docs/9.1/static/kernel-resources.html#SYSVIPC. Users reported that another way to go is to use a connection pooler like `PgBouncer `_. Services ======== @@ -34,7 +34,7 @@ Servers - *Symptom.* Some HTTP requests to ContestWebServer take a long time and fail with 500 Internal Server Error. ContestWebServer logs contain entries such as :samp:`TimeoutError('QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 60',)`. - *Possible cause.* The server may be overloaded with user requests. You can try to increase the ```pool_timeout``` argument in :gh_blob:`cms/db/__init__.py` or, preferably, spread your users over more instances of ContestWebServer. + *Possible cause.* The server may be overloaded with user requests. You can try to increase the ``pool_timeout`` argument in :gh_blob:`cms/db/__init__.py` or, preferably, spread your users over more instances of ContestWebServer. - *Symptom.* Message from ContestWebServer such as: :samp:`WARNING:root:Invalid cookie signature KFZzdW5kdWRlCnAwCkkxMzI5MzQzNzIwCnRw...` @@ -42,7 +42,7 @@ Servers - *Symptom.* Ranking Web Server displays wrong data, or too much data. - *Possible cause.* RWS is designed to handle groups of contests, so it retains data about past contests. If you want to delete previous data, run RWS with the ```-d``` option. See :doc:`RankingWebServer` for more details + *Possible cause.* RWS is designed to handle groups of contests, so it retains data about past contests. If you want to delete previous data, run RWS with the ``-d`` option. See :doc:`RankingWebServer` for more details. - *Symptom.* Ranking Web Server prints an "Inconsistent data" exception. @@ -53,7 +53,7 @@ Sandbox - *Symptom.* The Worker fails to evaluate a submission logging about an invalid (empty) output from the manager. - *Possible cause.* You might have been used a non-statically linked checker. The sandbox prevent dynamically linked executables to work. Try compiling the checker with ```-static```. Also, make sure that the checker was compiled for the architecture of the workers (e.g., 32 or 64 bits). + *Possible cause.* You might have been used a non-statically linked checker. The sandbox prevent dynamically linked executables to work. Try compiling the checker with ``-static``. Also, make sure that the checker was compiled for the architecture of the workers (e.g., 32 or 64 bits). - *Symptom.* The Worker fails to evaluate a submission with a generic failure.