How to Upload Folders to Gg Servers
Want to chat with us ? Telegram aqueduct : https://t.me/plik_rootgg
Plik
Plik is a scalable & friendly temporary file upload system (Wetransfer like) in golang.
Main features
- Powerful command line client
- Easy to use web UI
- Multiple data backend : File, OpenStack Swift, S3, Google Cloud Storage
- Multiple metadata backend : Sqlite3, PostgreSQL, MySQL
- OneShot : Files are destructed after the first download
- Stream : Files are streamed from the uploader to the downloader (nix stored server side)
- Removable : Give the ability to the uploader to remove files at any time
- TTL : Custom expiration date
- Password : Protect upload with login/pasgisword (Auth Basic)
- Comments : Add custom message (in Markdown format)
- User authentication : Local / Google / OVH
- Upload restriction : Source IP / Token
- Administrator CLI and web UI
- Server side encryption (with S3 data backend)
- ShareX Uploader : Directly integrated into ShareX
- plikSharp : A .NET API customer for Plik
- Filelink for Plik : Thunderbird Addon to upload attachments to Plik
Content Table
- Installation
- Configuration
- Data Backends
- Metadata Backends
- Customer CLI
- Become Client
- HTTP API
- Admin CLI
- Authentication
- Security
- Cross Compilation
- FAQ
- How To Contribute
Installation
From release
To run plik, information technology'south very simple :
$ wget https://github.com/root-gg/plik/releases/download/i.3.v/plik-ane.iii.5-linux-amd64.tar.gz $ tar xzvf plik-1.3.5-linux-amd64.tar.gz $ cd plik-i.three.five-linux-amd64/server $ ./plikd Et voilà ! You at present accept a fully functional instance of Plik running on http://127.0.0.1:8080.
You tin edit server/plikd.cfg to conform the configuration to your needs (ports, ssl, ttl, backend params,...)
From sources
To compile plik from sources, you'll demand golang and npm installed on your organisation
Git clone or become become the project and merely run brand :
$ make $ cd server && ./plikd Docker
Plik comes with multiarch docker images built for linux amd64/i386/arm/arm64:
- rootgg/plik:latest (latest release)
- rootgg/plik:{version} (release)
- rootgg/plik:dev (latest commit of master)
Meet the Plik Docker reference
Plik as well comes with some useful scripts to test backend in standalone docker instances :
Come across the Plik Docker backend testing
Configuration
The configuration is managed using a TOML file plikd.cfg
Defining configuration parameters using environs variables
I tin can specify configuration parameters using env variable with the configuration parameter in screaming ophidian case
PLIKD_DEBUG_REQUESTS=true ./plikd For Arrays and config maps they must be provided in json format. Arrays are overridden just maps are merged
PLIKD_DATA_BACKEND_CONFIG='{"Directory":"/var/files"}' ./plikd Information backends
Plik is shipped with multiple data backend for uploaded files and metadata backend for the upload metadata.
- File databackend :
Store uploaded files in a local or mounted file system directory.
- Openstack Swift databackend : http://docs.openstack.org/developer/swift/
Openstack Swift is a highly available, distributed, eventually consequent object/hulk shop which supports Server Side Encryption
-
Amazon S3
-
Google Cloud Storage
Metadata backends
- Sqlite3
Suitable for standalone deployment.
- PostgreSQL / Mysql
Suitable for distributed / High Availability deployment.
Cli client
Plik is shipped with a powerful golang multiplatform cli client (downloadable in spider web interface) :
Usage: plik [options] [FILE] ... Options: -h --help Show this assist -d --debug Enable debug mode -q --quiet Enable quiet mode -o, --oneshot Enable OneShot ( Each file will be deleted on first download ) -r, --removable Enable Removable upload ( Each file can be deleted by anyone at anymoment ) -Due south, --stream Enable Streaming ( It will block until remote user starts downloading ) -t, --ttl TTL Time earlier expiration (Upload will exist removed in m|h|d) -n, --name Proper noun Set file proper name when piping from STDIN --server SERVER Overrides plik url --token TOKEN Specify an upload token --comments Annotate Set comments of the upload ( MarkDown compatible ) -p Protect the upload with login and countersign --countersign PASSWD Protect the upload with login:password ( if omitted default login is "plik" ) -a Archive upload using default annal params ( see ~/.plikrc ) --archive Way Archive upload using specified archive backend : tar|zip --compress Manner [tar] Pinch codec : gzip|bzip2|xz|lzip|lzma|lzop|compress|no --archive-options OPTIONS [tar|zero] Additional command line options -s Encrypt upload usnig default encrypt params ( see ~/.plikrc ) --not-secure Practice not encrypt upload regardless of ~/.plikrc configurations --secure MODE Archive upload using specified archive backend : openssl|pgp --aught CIPHER [openssl] Openssl aught to apply ( see openssl help ) --passphrase PASSPHRASE [openssl] Passphrase or '-' to be prompted for a passphrase --recipient RECIPIENT [pgp] Set recipient for pgp backend ( example : --recipient Bob ) --secure-options OPTIONS [openssl|pgp] Additional control line options --update Update client -v --version Prove client version For instance to create directory tar.gz annal and encrypt it with openssl :
$ plik -a -s mydirectory/ Passphrase : 30ICoKdFeoKaKNdnFf36n0kMH Upload successfully created : https://127.0.0.1:8080/ #/?id=0KfNj6eMb93ilCrl mydirectory.tar.gz : fifteen.70 MB 5.92 MB/due south Commands : curl -south 'https://127.0.0.ane:8080/file/0KfNj6eMb93ilCrl/q73tEBEqM04b22GP/mydirectory.tar.gz' | openssl aes-256-cbc -d -pass pass:30ICoKdFeoKaKNdnFf36n0kMH | tar xvf - --gzip Customer configuration and preferences are stored at ~/.plikrc or /etc/plik/plikrc ( overridable with PLIKRC environement variable )
Quick upload using curl only
whorl --grade 'file=@/path/to/file' http://127.0.0.1:8080 When Authentication is used and NoAnonymousUploads are enabled yous can quick upload using user tokens:
gyre --grade 'file=@/path/to/file' --header 'Ten-PlikToken: xxxx-xxx-xxxx-xxxxx-xxxxxxxx' http://127.0.0.1:8080 DownloadDomain configuration option must be fix for this to properly piece of work.
Go client
Plik at present comes with a golang library above which the cli client is built
See the Plik library reference
API
Plik server expose a HTTP API to manage uploads and get files :
See the Plik API reference
Admin CLI
Using the ./plikd server binary it'southward possible to :
- create/list/delete local accounts
- create/list/delete user CLI tokens
- create/list/delete files and uploads
- import / export metadata
Encounter help for more details
Authentication
Plik can cosign users using Local accounts or using Google or OVH APIs.
If source IP address brake is enabled, user accounts tin can simply be created from trusted IPs so authenticated users tin upload files without source IP brake.
It possible to deny unauthenticated uploads totally ( NoAnonymousUploads ).
Admin users can access the admin dashboard and manipulate every uploads.
-
Local :
- You can dispense local users with the server control line
$ ./plikd --config ./plikd.cfg user create --login root --name Admin --admin Generated countersign for user root is 08ybEyh2KkiMho8dzpdQaJZm78HmvWGC -
Google :
- You'll need to create a new application in the Google Developper Panel
- You'll exist handed a Google API ClientID and a Google API ClientSecret that you'll demand to put in the plikd.cfg file.
- Do non forget to whitelist valid origin and redirect url ( https://yourdomain/auth/google/callback ) for your domain.
- Information technology is possible to whitelist merely i or more e-mail domains.
-
OVH :
- You'll need to create a new application in the OVH API : https://eu.api.ovh.com/createApp/
- You'll be handed an OVH awarding central and an OVH application cloak-and-dagger key that you'll need to put in the plikd.cfg file.
Once authenticated a user can generate upload tokens that tin can be specified in the ~/.plikrc file to authenticate the control line client.
Token = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" Security
Plik permit users to upload and serve any content equally-is, simply hosting untrusted HTML raises some well known security concerns.
Plik will try to avoid HTML rendering by overriding Content-Type to "text-evidently" instead of "text/html".
By default Plik sets a couple of security HTTP headers like X-Content-Blazon-Options, X-XSS-Protection, X-Frame-Options, Content-Security-Policy to disable sensible features of most recent browsers similar resource loading, xhr requests, iframes,... This will however suspension features like audio/video playback, pdf rendering so it's possible to disable this behavior by setting the EnhancedWebSecurity configuration parameter to false
Forth with that it is as well strongly advised to serve uploaded files on a split (sub-)domain to fight confronting phishing links and to protect Plik's session cookie with the DownloadDomain configuration parameter.
Cross compilation
All binary are now statically linked. Clients tin can be safely cross-compiled for all bone/architectures as they practise not rely on GCO (sqlite) Servers rely on CGO/sqlite need a cantankerous-compilation gear up environment.
make release volition build release archives for amd64,i386,arm,arm64
To build a release with only specific architectures of the client
CLIENT_TARGETS="linux/amd64" releaser/release.sh To build only specific for only specific architectures
TARGETS="linux/amd64" releaser/release.sh To build with a specific cross compiler toolchain
TARGETS="linux/arm/v6" CC=arm-linux-gnueabihf-gcc releaser/release.sh FAQ
- Why is stream manner cleaved in multiple instance deployement ?
Beacause stream style isn't stateless. As the uploader asking will block on 1 plik case the downloader request MUST get to the same instance to succeed. The load balancing strategy MUST be aware of this and route stream requests to the same instance by hashing the file id.
Hither is an instance of how to achieve this using nginx and a picayune piece of LUA. Make sure your nginx server is built with LUA scripting support. You might want to install the "nginx-extras" Debian bundle (>ane.7.two) with built-in LUA back up.
upstream plik { server 127.0.0.1:8080; server 127.0.0.ane:8081; } upstream stream { server 127.0.0.1:8080; server 127.0.0.1:8081; hash $hash_key; } server { mind 9000; location / { set $upstream ""; gear up $hash_key ""; access_by_lua ' _,_,file_id = string.find(ngx.var.request_uri, "^/stream/[a-zA-Z0-nine]+/([a-zA-Z0-ix]+)/.*$") if file_id == nil then ngx.var.upstream = "plik" else ngx.var.upstream = "stream" ngx.var.hash_key = file_id end '; proxy_pass http://$upstream; } } - Redirection loops with DownloadDomain enforcement and reverse proxy
Invalid download domain 127.0.0.i:8080, expected plik.root.gg DownloadDomain check the Host header of the incoming HTTP request, by default reverse proxies similar Nginx or Apache mod_proxy does not forwards this Header. Bank check the following configuration directive :
Apache mod_proxy : ProxyPreserveHost On Nginx : proxy_set_header Host $host; - I take an error when uploading from client : "Unable to upload file : HTTP error 411 Length Required"
Nether nginx < 1.3.9, you must enable HttpChunkin module to allow transfer-encoding "chunked".
You might want to install the "nginx-extras" Debian packet with congenital-in HttpChunkin module.
And add together in your server configuration :
chunkin on; error_page 411 = @my_411_error; location @my_411_error { chunkin_resume; } - How to disable nginx buffering ?
By default nginx buffers big HTTP requests and reponses to a temporary file. This behaviour leads to unnecessary disk load and slower transfers. This should be turned off (>1.vii.12) for /file and /stream paths. You might as well want to increase buffers size.
Detailed documentation : http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
proxy_buffering off; proxy_request_buffering off; proxy_http_version 1.1; proxy_buffer_size 1M; proxy_buffers 8 1M; client_body_buffer_size 1M; - Why authentication does not work with HTTP connections when EnhancedWebSecurity is set ?
Plik session cookies take the "secure" flag set when EnhancedWebSecurity is set and so they tin merely be transmitted over secure HTTPS connections.
- Build failure "/usr/bin/env: 'node': No such file or directory"
Debian users might need to install the nodejs-legacy parcel.
This bundle contains a symlink for legacy Node.js code requiring binary to be /usr/bin/node (not /usr/bin/nodejs as provided in Debian). - How to take and upload screenshots like a dominate ?
allonym pshot="scrot -southward -e 'plik -q \$f | xclip ; xclip -o ; rm \$f'" Requires you lot to accept plik, scrot and xclip installed in your $PATH.
scrot -due south allow you to "Interactively select a window or rectangle with the mouse" then Plik will upload the screenshot and the url volition be direct copied to your clipboard and displayed by xclip. The screenshot is and so removed of your home directory to avoid garbage.
How to contribute to the project ?
Contributions are welcome, experience gratis to open up issues and/or submit pull requests. Please be sure to likewise run/update the test suite :
brand fmt make lint brand test make test-backends Source: https://github.com/root-gg/plik
0 Response to "How to Upload Folders to Gg Servers"
Post a Comment