Everything GPT-2: 7. Production

  1. AWS — Free account with administrative username setup for programmatic access. When you setup the account, you will create a root user. Use the root user to setup an administrator with programmatic access. Use this one for Cortex and your own use. Set up at least one empty S3 Storage bucket.
  2. Docker — Installed.
  3. Cortex — Installed per their website. Cortex basic tutorials completed. If you are running windows, then check out their Windows installation guide for running Cortex using WSL (Windows Subsystem for Linux) version 2. Note: If you install Ubuntu 20.04 (recommended), then you will have to run sudo apt-get install python3-pip and pip3 install cortex from the WSL Bash window because Ubuntu 20.04 does not ship with pip3.
  4. AWS Command Line Interface Version 2 — Installed. If using Windows, installed on WSL.
  5. Git Command Line — Installed.
  1. Configure your AWS credentials with the command line utility. Do this on the OS which will be running Cortex. I tried configuring my credentials multiple ways, and I could not get my credentials to work. Configuring with the command line utility was very straight forward. Simply run aws configure and set your configuration. Set your Default output format: json. Make sure that the S3 bucket that you will use is in the same region as your default region otherwise you will incur extra fees from inter-region transfers.
  2. Upload your model and tokenizer to AWS S3 with this Colab Notebook
  3. Clone the GPT-2 Everything repo to your primary coding machine
git clone https://github.com/Rowlando13/EverythingGPT-2.git
cp -r /mnt/c/your_source_directories/EverythingGPT-2/cortex your_destination_directories
cp -r /mnt/c/your_source_directories/EverythingGPT-2/api_testing_samples your_destination_directories
cd ~/EverythingGPT-2/cortex
cortex deploy
cortex logs gpt2-774m-pulitzer
2020-12-15 21:11:37.039702:cortex:pid-242:INFO:Application startup complete.
2020-12-15 21:11:37.040991:cortex:pid-242:INFO:Uvicorn running on unix socket /run/uvicorn/proc-0.sock (Press CTRL+C to quit)
cd ..
cd api_testing_samples
curl http://localhost:8890 -X POST -H "Content-Type: application/json" -d @sample1.json
{"text": {"0": " first alternate completion to you sentence.", "1": " second alternate completion to your sentence."}, "truncated": false}
cortex delete gpt2-774m-pulitzer
cd .. 
cd cortex
cortex cluster up --config cluster1.yaml
cortex env default aws
cortex deploy
cortex logs gpt2-774m-pulitzer
cd ..
cd api_testing_samples
curl yourapiendpoint -X POST -H "Content-Type: application/json" -d @sample1.json
cortex cluster down

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Edward Girling

Edward Girling

Mathematician, enjoys his knowledge distilled. Find my insight deep, my jokes laughable, my resources useful, connect with me on twitter @Rowlando_13