AI Configuration

Configure AI Inference Services on a Livepeer Gateway

Page is under construction.
Feedback Welcome!

This page will walk you through the process of deploying and configuring a Livepeer Gateway for AI inference services.

Gateway Modes

You can run a gateway

• “Off-chain” -> dev or local mode
• “On-chain” -> production mode connected to the blockchain-based Livepeer network.

If you run your gateway off-chain - you will need to run your own Orchestrator node ie. have access to a GPU and set it up as an Orchestrator, in order to test Gateway functionality.

There is currently no Livepeer “Testnet” available which has Orchestrator offerings, though there are conversations underway to enable this in the future.

Do you think Livepeer should have a “testnet” available for Gateways to connect to?

Follow & contribute to the discussion in the Discord and on the Forum)

Deploy a Gateway for AI Inference Services

You can run the Livepeer AI software using one of the following methods:

• Docker (Recommended): The simplest and preferred method.
• Pre-built Binaries: An alternative if you prefer not to use Docker.

Deploy an AI Gateway

Follow the steps below to start your Livepeer AI Gateway node.

These instructions apply to both on-chain & off-chain Gateway deployments.

Use Docker (Recommended)

1. Retrieve the Livepeer AI Docker Image

   Fetch the latest Livepeer AI Docker image with the following command:

   Pull Docker Image

   docker pull livepeer/go-livepeer:master
   
2. Launch a Localhost (Off-chain) AI Gateway

   Run the Docker container for your AI Gateway node:

   docker run \
       --name livepeer_ai_gateway \
       -v ~/.lpData2/:/root/.lpData2 \
       -p 8937:8937 \
       --network host \
       livepeer/go-livepeer:master \
       -datadir ~/.lpData2 \
       -gateway \
       -orchAddr <orchestrator list> \
       -httpAddr 0.0.0.0:8937 \
       -v 6 \
       -httpIngest
   

   This command launches a local (off-chain) AI Gateway node. The flags are similar to those used for a Mainnet Transcoding Network Gateway.

3. Confirm Successful Startup

   Upon successful startup, you should see output similar to:

   I0501 11:07:47.609839       1 mediaserver.go:201] Transcode Job Type: [{P240p30fps16x9 600k 30 0 426x240 16:9 0 0 0s 0 0 0 0} {P360p30fps16x9 1200k 30 0 640x360 16:9 0 0 0s 0 0 0 0}]
   I0501 11:07:47.609917       1 mediaserver.go:226] HTTP Server listening on http://0.0.0.0:8937
   I0501 11:07:47.609963       1 lpms.go:92] LPMS Server listening on rtmp://127.0.0.1:1935
   
4. Check Port Availability

   Ensure that port 8937 is open and accessible, and configure your router for port forwarding if necessary to make the Gateway accessible from the internet.

5. Test the AI Gateway

   Follow the instructions in the AI Gateway Testing Guide to verify that the AI Gateway is operational.

Use Binaries

1. Download the Latest Livepeer AI Binary

   Download the latest Livepeer AI binary for your system:

   wget https://build.livepeer.live/go-livepeer/livepeer-<OS>-<ARCH>.tar.gz
   

   Replace <OS> and <ARCH> with your operating system and architecture (e.g., linux-amd64 for Linux AMD64). For more details, see the go-livepeer installation guide.

   The Windows and MacOS (amd64) binaries of Livepeer AI are not available yet.
2. Extract and Configure the Binary

   Once downloaded, extract the binary to a directory of your choice.

3. Launch an Off-chain AI Gateway

   Start the AI Gateway node with the following command:

   ./livepeer \
       -datadir ~/.lpData2 \
       -gateway \
       -orchAddr <orchestrator list> \
       -httpAddr 0.0.0.0:8937 \
       -v 6 \
       -httpIngest
   

   This command launches an off-chain AI Gateway node. Refer to the go-livepeer CLI reference for more details on the flags.

4. Confirm Successful Startup

   Check the terminal for the following output to confirm successful startup:

   I0501 11:07:47.609839       1 mediaserver.go:201] Transcode Job Type: [{P240p30fps16x9 600k 30 0 426x240 16:9 0 0 0s 0 0 0 0} {P360p30fps16x9 1200k 30 0 640x360 16:9 0 0 0s 0 0 0 0}]
   I0501 11:07:47.609917       1 mediaserver.go:226] HTTP Server listening on http://0.0.0.0:8937
   I0501 11:07:47.609963       1 lpms.go:92] LPMS Server listening on rtmp://127.0.0.1:1935
   
5. Check Port Availability

   Ensure that port 8937 is open and accessible, and configure port forwarding if needed.

6. Test the AI Gateway

   Follow the instructions in the AI Gateway Testing Guide to verify that the AI Gateway is operational.

If binaries are unavailable for your system, you can build the master branch of go-livepeer from source. Refer to the Livepeer installation guide or reach out to the Livepeer community on Discord for assistance.

Gateway Code Links

For AI processing, the Gateway extends its functionality to handle AI-specific workflows.

go-livepeer/server/ai_mediaserver.go

Key components include:

• AISessionManager: Manages AI processing sessions and selects appropriate Orchestrators with AI capabilities ai_http.go
• MediaMTX Integration: Handles media streaming for AI processing
• Trickle Protocol: Enables efficient streaming for real-time AI video processing

The AI workflow server/ai_process.go involves

• authenticating AI streams,
• selecting AI-capable Orchestrators,
• processing payments based on pixels, and
• managing live AI pipelines