s3-satis tool

Tool to generate a Composer PHP packages repository (based on Satis - static Composer repository generator) and synchronize generated repository with a Amazon S3 (or compatible) bucket.

GitHub RepositoryDownload s3-satis.phar

Setup

You can install s3-satis tool in four ways:

Docker
docker run \  --volume "${PWD}/satis.json:/usr/src/satis/satis.json" \  --volume "${COMPOSER_HOME:-$HOME/.composer}:/root/.composer/" \  --env S3_ACCESS_KEY_ID=<S3_ACCESS_KEY_ID> \  --env S3_SECRET_ACCESS_KEY=<S3_SECRET_ACCESS_KEY> \  --env S3_REGION=<S3_REGION> \  --env S3_BUCKET=<S3_BUCKET> \  --env S3_ENDPOINT=<S3_ENDPOINT> \  --env S3_USE_PATH_STYLE_ENDPOINT=<S3_USE_PATH_STYLE_ENDPOINT> \  ghcr.io/kduma-oss/s3-satis:latest build /usr/src/satis/satis.json
Composer Global
composer global require kduma/s3-satis
phar executable
wget https://github.com/kduma-OSS/CLI-pdf-scan-splitter/releases/latest/download/s3-satis.pharchmod +x s3-satis.pharmv s3-satis.phar /usr/local/bin/s3-satis
Sources
git clone https://github.com/kduma-OSS/CLI-pdf-scan-splitter.git s3-satiscd s3-satiscomposer install

Requirements

You need to have prepared a S3 (or compatible) bucket with a provider of your choice, and generated an access key and secret key with write access for created bucket.

Usually you need also to have authorized your composer installation to access your private repositories. If you want to create a repository consisting of publicly available packages on GitHub, you will probably also need to generate a GitHub access token, because of GitHub API rate limits.

Usage

First prepare a satis.json file with your repository configuration. This tool is based on Satis - static Composer repository generator, so please check Satis documentation for configuration options.

Simple satis.json example
{  "name": "my/repo",  "homepage": "https://satis.example.com",  "repositories": [    { "type": "vcs", "url": "https://github.com/laravel/framework" }  ],  "require-all": true}
Simple satis.json example with archiving
{  "name": "my/repo",  "homepage": "https://satis.example.com",  "repositories": [    { "type": "vcs", "url": "https://github.com/laravel/framework" }  ],  "require-all": true,  "archive": {    "directory": "dist"  }}

Second, configure your environment variables (or .env file) with your S3 bucket credentials:

S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
S3_REGION=us-east-1
S3_BUCKET=
S3_ENDPOINT=
S3_USE_PATH_STYLE_ENDPOINT=false

Then run s3-satis tool to generate repository and upload it to S3 bucket:

s3-satis build satis.json

Extensions

s3-satis tool provides a few extensions to Satis functionality. You can enable them by adding them to your satis.json file, in s3-satis.plugins section, or by providing them as a command line argument like: s3-satis build --plugin=my_plugin_name:param1=value1:param2=value2.

In satis.json you can provide them like this:

{  "s3-satis": {    "plugins": {      "my_plugin_name": {        "enabled": true,        "param1": "value1",        "param2": "value2"      }    }  }}

or, if you don't need to provide any parameters, you can omit configuration and only provide boolean value:

{  "s3-satis": {    "plugins": {      "my_plugin_name": true    }  }}

Cache Extension: cache

This extension adds an ability to cache downloaded metadata files in local directory and not download them from S3 bucket on every run. In path parameter you can provide a path to directory where cache files will be stored and retrieved, and if you provide temp as a value - files will be stored in system temp directory. If you provide copy parameter as true, files will be copied from cache directory, otherwise they will be moved - so if for any reason script crashes during execution, the cache will be left empty.

{  "s3-satis": {    "plugins": {      "cache": {        "enabled": true,        "path": "temp",        "copy": false      }    }  }}

File Restrictions Map Generator Extension: file-restrictions-map-generator

This extension will generate a files in repository root directory, which will contain a version and package identifiers of archives in repository.

It there is a file dist/vendor/package/package-xxxxxxx.tar generated, and it has a version of 1.0.0.0 in corresponding packages.json file, there will be generated .tags/dist/vendor/package/package-xxxxxxx.tar.json file with following content:

[  "vendor/package:1.0.0.0",  "vendor/package:1.0.0.x",  "vendor/package:1.0.x",  "vendor/package:1.x"]

Or if it is not released version, but a branch, the file will be generated with dev-{branch} tag:

[  "vendor/package:dev-master"]

You can use this extension with CloudFlare Worker to provide selective access to packages in your private repository.

If you enable extra-json option, generated tags will be added to extra section of Composer 2 packages.json file, so you can filter them out in CloudFlare Worker

{  "s3-satis": {    "plugins": {      "file-restrictions-map-generator": {        "enabled": true,        "extra-json": false      }    }  }}

Remove Fields From Json Extension: remove-fields-from-json

When enabled, this extension will remove specified fields from exported json files. Only fields listed bellow are supported, other fields will be ignored.

{  "s3-satis": {    "plugins": {      "remove-fields-from-json": {        "enabled": true,        "remove": [          "source",          "authors",          "homepage",          "support"        ]      }    }  }}

Skip Remote Versions Extension: skip-remote-versions

When enabled, this extension will filter out all entries in exported json files which doesn't provide local download link (e.g. are linked directly to GitHub repository)

{  "s3-satis": {    "plugins": {      "skip-remote-versions": true    }  }}

Debugging Extensions

There are few extensions which can be used for debugging purposes:

Skip Step After Hooks Extension: skip-step-after-hook

Allows to skip execution of specified builtin steps.

{  "s3-satis": {    "plugins": {      "skip-step-after-hook": {        "enabled": true,        "skip": [          "BEFORE_INITIAL_CLEAR_TEMP_DIRECTORY",          "BEFORE_CREATE_TEMP_DIRECTORY",          "BEFORE_DOWNLOAD_FROM_S3",          "BEFORE_BUILD_SATIS_REPOSITORY",          "BEFORE_UPLOAD_TO_S3",          "BEFORE_REMOVE_MISSING_FILES_FROM_S3",          "BEFORE_FINAL_CLEAR_TEMP_DIRECTORY"        ]      }    }  }}

Pause At Hooks Extension: pause-at-hook

Allows to pause execution of at specified hook points. Press enter to continue execution.

{  "s3-satis": {    "plugins": {      "pause-at-hook": {        "enabled": true,        "pause": [          "BEFORE_INITIAL_CLEAR_TEMP_DIRECTORY",          "AFTER_INITIAL_CLEAR_TEMP_DIRECTORY",          "BEFORE_CREATE_TEMP_DIRECTORY",          "AFTER_CREATE_TEMP_DIRECTORY",          "BEFORE_DOWNLOAD_FROM_S3",          "AFTER_DOWNLOAD_FROM_S3",          "BEFORE_BUILD_SATIS_REPOSITORY",          "AFTER_BUILD_SATIS_REPOSITORY",          "BEFORE_UPLOAD_TO_S3",          "AFTER_UPLOAD_TO_S3",          "BEFORE_REMOVE_MISSING_FILES_FROM_S3",          "AFTER_REMOVE_MISSING_FILES_FROM_S3",          "BEFORE_FINAL_CLEAR_TEMP_DIRECTORY",          "AFTER_FINAL_CLEAR_TEMP_DIRECTORY"        ]      }    }  }}
Table of Contents