Paste and file hosting service with Bash or Zsh and SSH

Published on 2019-06-09

This is a guide on setting up a self hosted paste or file hosting service with SSH, Bash or Zsh and your choice of webserver software.

You need

  • SSH access to a webserver,
  • Bash or Zsh installed on your local computer.

You can still use this guide if you do not have SSH access to your server (e.g. on shared hosting), but you will have to rewrite the shell code to upload files using a supported method (FTP, WebDAV, etc.).

Uploading files from the command line

Create a directory on the webserver (modify path as needed):

user@webserver ~$: mkdir -p /var/www/htdocs/example.com/files

It is probably not a bad idea to disable server side execution of dynamic content (PHP, CGI, SSI, etc.) for this directory. Depending on your use case you might want to configure your webserver software to serve all files with the text/plain MIME type, also.

Add an upload function to your .bashrc or .zshrc:

function upload() {
  # generate a random string
  FILE_ID=$(base64 /dev/urandom | tr -d '/+' | dd bs=16 count=1 2>/dev/null)

  # save STDIN to a file on the server
  ssh user@webserver.example.com "cat > /var/www/htdocs/example.com/files/$FILE_ID-$1"

  # generate the public URL
  FILE_URL="https://example.com/files/$FILE_ID-$1"

  # copy the URL to the clipboard
  # for macOS:
  # echo $FILE_URL | pbcopy
  # for (Linux) systems with xclip installed:
  # echo $FILE_URL | xclip -selection clipboard
  # for (Linux) systems with xsel installed:
  # echo $FILE_URL | xsel --clipboard --input

  # diplay the URL
  echo $FILE_URL
}

Using public key authentication in combination with ssh-agent does away with the password prompts.

Test whether the setup works:

user@local ~$: cat example.php | upload 'php-example.txt'
https://example.com/files/09nnX6LPjBxtO4HO-php-example.txt

user@local ~$: curl https://example.com/files/09nnX6LPjBxtO4HO-php-example.txt
<?php phpinfo(); ?>

Automatic expiration

To support automatic expiration of uploads, create one directory for every timespan you want to support:

# 1 day
user@webserver ~$: mkdir -p /var/www/htdocs/example.com/files/d
# 1 week
user@webserver ~$: mkdir -p /var/www/htdocs/example.com/files/w
# 1 month
user@webserver ~$: mkdir -p /var/www/htdocs/example.com/files/m
# eternity
user@webserver ~$: mkdir -p /var/www/htdocs/example.com/files/e

Add cronjobs to your crontab to remove expired files:

# delete all files in the d directory older than 1 day
* * * * * /usr/bin/find /var/www/htdocs/example.com/files/d -type f -mtime +1  -exec rm {} \;

# delete all files in the w directory older than 7 days or 1 week
* * * * * /usr/bin/find /var/www/htdocs/example.com/files/w -type f -mtime +7  -exec rm {} \;

# delete all files in the m directory older than 30 days or 1 "month"
* * * * * /usr/bin/find /var/www/htdocs/example.com/files/m -type f -mtime +30 -exec rm {} \;

Extend the shell function:

function upload() {
  # make eternity the default lifespan
  FILE_LIFESPAN=${2:-e}

  FILE_ID=$(base64 /dev/urandom | tr -d '/+' | dd bs=16 count=1 2>/dev/null)

  ssh user@webserver.example.com "cat > /var/www/htdocs/example.com/files/$FILE_LIFESPAN/$FILE_ID-$1"

  FILE_URL="https://example.com/files/$FILE_LIFESPAN/$FILE_ID-$1"

  # echo $FILE_URL | pbcopy
  # echo $FILE_URL | xclip -selection clipboard
  # echo $FILE_URL | xsel --clipboard --input

  echo $FILE_URL
}

Test it:

user@local ~$: cat picture.jpg | upload 'cat-picture.jpg' d
https://example.com/files/d/xuuUdmWvZuFnZM82-cat-picture.jpg

user@local ~$: curl -I https://example.com/files/d/xuuUdmWvZuFnZM82-cat-picture.jpg
HTTP/1.1 200 OK
[...]

user@local ~$: cat example.php | upload 'php-example.txt'
https://example.com/files/e/19nnX6LPjBxtO4HO-php-example.txt

user@local ~$: curl -I https://example.com/files/e/19nnX6LPjBxtO4HO-php-example.txt
HTTP/1.1 200 OK
[...]

Check up on your files after 24 hours:

user@local ~$: curl -I https://example.com/files/d/xuuUdmWvZuFnZM82-cat-picture.jpg
HTTP/1.1 404 Not Found
[...]

user@local ~$: curl -I https://example.com/files/e/19nnX6LPjBxtO4HO-php-example.txt
HTTP/1.1 200 OK
[...]

You are done.