Artix Linux website code
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Anton Hvornum 0357225af2 Fixed flake8 issue 5 months ago
.github Update Django to 4.0 7 months ago
devel Mutualize commands logger configs 6 months ago
docs Add Tier 0 Mirror Access support 1 year ago
main Mutualize commands logger configs 6 months ago
mirrors Mutualize commands logger configs 6 months ago
news Replaced re_path with path where path can be used 7 months ago
packages Fixed flake8 issue 5 months ago
planet Mutualize commands logger configs 6 months ago
public Introduce planet functionality in archweb 2 years ago
releng Add sha256 and b2sum to release page 6 months ago
sitestatic Make the RSS icon orange and optimize it's size 8 months ago
templates remove aur.archlinux.org .php extension usage 5 months ago
todolists Replaced re_path with path where path can be used 7 months ago
visualize Replaced re_path with path where path can be used 7 months ago
.coveragerc Update coverage's ignore file 10 months ago
.flake8 Remove flake8 W503 issues 9 months ago
.gitattributes Add gitattributes file 12 years ago
.gitignore Added a JSON search endpoint for groups. The result from the group search is all packages that belong to a specific group. 5 months ago
.gitlab-ci.yml Update docker container image name 1 year ago
.pyup.yml create pyup.io config file 4 years ago
AUTHORS Add myself as maintainer 4 years ago
HACKING update documentation 14 years ago
LICENSE Initial import for public release... 15 years ago
Makefile Add a makefile for running tests 10 months ago
README Move markdown formatted readme to README.md 9 years ago
README.md Document new testing setup 10 months ago
archweb.wsgi Fix hardcoded python version in archweb.wsgi 3 years ago
conftest.py Use DummyCache in tests 6 months ago
feeds.py Add author to planet feed 6 months ago
local_settings.py.example Switch to pymemcache 10 months ago
manage.py Switch shebang for Python 3 support 3 years ago
requirements.txt Update django to 4.0.2 5 months ago
requirements_prod.txt Switch to pymemcache 10 months ago
requirements_test.txt pylint is no longer used, flake8 replaced it 7 months ago
settings.py Mutualize commands logger configs 6 months ago
sitemaps.py Fix pep8 errors 2 years ago
urls.py Replaced re_path with path where path can be used 7 months ago

README.md

Archweb README

Build Status

To get a pretty version of this document, run

$ markdown README > README.html

License

See LICENSE file.

Authors

See AUTHORS file.

Dependencies

  • python
  • rsync (optional for mirrorcheck with rsync mirrors)

Python dependencies

More detail in requirements.txt and requirements_prod.txt; it is best to use virtualenv and pip to handle these. But if you insist on (Arch Linux) packages, you will probably want the following:

  • python-django
  • python-psycopg2
  • python-markdown
  • python-memcached

Testing Installation

  1. Run python -m venv env.

     cd /path/to/archweb && python -m venv ./env/
    
  2. Activate the virtualenv.

     source ./env/bin/activate
    
  3. Install dependencies through pip.

     pip install -r requirements.txt
    
  4. Copy local_settings.py.example to local_settings.py and modify. Make sure to uncomment the appropriate database section (either sqlite or PostgreSQL).

  5. Migrate changes.

     ./manage.py migrate
    
  6. Load the fixtures to pre populate some data. If you don't want some of the provided data, adjust the file glob accordingly.

     ./manage.py loaddata main/fixtures/*.json
     ./manage.py loaddata devel/fixtures/*.json
     ./manage.py loaddata mirrors/fixtures/*.json
     ./manage.py loaddata releng/fixtures/*.json
    
  7. Use the following commands to start a service instance

     ./manage.py runserver
    
  8. To optionally populate the database with real data:

     wget http://mirrors.kernel.org/archlinux/core/os/x86_64/core.db.tar.gz
     ./manage.py reporead x86_64 core.db.tar.gz
     # Package file listing
     wget http://mirrors.kernel.org/archlinux/core/os/x86_64/core.files.tar.gz
     ./manage.py reporead --filesonly x86_64 core.files.tar.gz
    

Alter architecture and repo to get x86_64 and packages from other repos if needed.

  1. Database Updates for Added/Removed packages

     sqlite3 archweb.db < packages/sql/update.sqlite3.sql
    

For PostgreSQL use packages/sql/update.postgresql_psycopg2.sql

Testing SMTP server

To be able to create an account on your test environment an SMTP server is required. A simple debugging SMTP server can be setup using Python.

    python -m smtpd -n -c DebuggingServer localhost:1025

In local_settings.py add entries to set EMAIL_HOST to 'localhost' and EMAIL_PORT to 1025.

Running tests and coverage

Install the test dependencies:

    pip install -r requirements_test.txt

To the unittests execute the following commands:

    make collectstatic
    make test

Running coverage:

    make coverage
    make open-coverage

Django Debug toolbar

To use the Django Debug toolbar install django-debug-toolbar and in local_settings.py set DEBUG_TOOLBAR to True.

Management commands

Archweb provides multiple management commands for importing various sorts of data. An overview of commands:

  • generate_keyring - Assemble a GPG keyring with all known developer keys.
  • pgp_import - Import keys and signatures from a given GPG keyring.
  • read_rebuilderd_status - Import rebuilderd status into Archweb.
  • rematch_developers - Rematch flag requests and packages where user_id/packager_id is NULL to a Developer.
  • reporead - Parses a repo.db.tar.gz, repo.files.tar.gz file and updates the Arch database with the relevant changes.
  • reporead_inotify - Watches a templated patch for updates of *.files.tar.gz to update Arch databases with.
  • donor_import - Import a single donator from a mail passed to stdin
  • mirrorcheck - Poll every active mirror URLs to store the lastsnyc time and record network timing details.
  • mirrorresolv - Poll every active mirror URLs and determine wheteher they have IP4 and/or IPv6 addresses.
  • populate_signoffs - retrieves the latest commit message of a signoff-eligible package.
  • update_planet - Import all feeds for users who have a valid website and website_rss in their user profile.
  • read_links - Reads a repo.links.db.tar.gz file and updates the Soname model.
  • read_links_inotify - Watches a templated patch for updates of *.links.tar.gz to update Arch databases with.

Updating iPXE images

The binaries required for iPXE based netboot are updated by copying them from the ipxe package to the static content directory (with the run_ipxe script the binaries may be tested beforehand):

cp -v /usr/share/ipxe/x86_64/ipxe-arch.efi /usr/share/ipxe/ipxe-arch.{ipxe,lkrn} sitestatic/releng

Afterwards a detached PGP signature using a valid WKD enabled packager key is created for each file:

gpg --sender "User Name <your@mail.address>" --detach-sign sitestatic/netboot/*.{efi,ipxe,lkrn}

Production Installation

Arch Linux has an Ansible role for Archweb in their infrastructure repo.

vim: set syntax=markdown et: