Self-Hosted Pixie

Get Pixie fully managed with Pixie Community Cloud (free forever) or run on your own infrastructure with the following self-managed option.

• Review Pixie's requirements to make sure that your Kubernetes cluster is supported.

• Determine if you already have Operator Lifecycle Manager (OLM) deployed to your cluster, possibly to the default olm namespace. Pixie uses the Kubernetes Operator pattern to manage its Vizier, which handles data collection and query execution (see the Architecture diagram). The OLM is used to install, update and manage the Vizier Operator.

• Ensure that your cluster supports Pixie creating and using PersistentVolumes.

• Pixie interacts with the Linux kernel to install BPF programs to collect telemetry data. In order to install BPF programs, Pixie vizier-pem-* pods require privileged access.

1. Clone the Pixie repo.

   git clone https://github.com/pixie-io/pixie.git
   cd pixie
   

2. Pick a cloud release version from the tags on the repo. The following should pick the latest release for you.

   export LATEST_CLOUD_RELEASE=$(git tag | grep 'release/cloud'  | sort -r | head -n 1 | awk -F/ '{print $NF}')
   

3. Checkout the release tag.

   git checkout "release/cloud/prod/${LATEST_CLOUD_RELEASE}"
   

4. Update the versions in the appropriate kustomization file.

   perl -pi -e "s|newTag: latest|newTag: \"${LATEST_CLOUD_RELEASE}\"|g" k8s/cloud/public/kustomization.yaml
   

5. (Optional) By default, the self-hosted Pixie Cloud will be accessible through dev.withpixie.dev. If you wish to use a custom domain name, replace all occurances of dev.withpixie.dev in the following files with the domain name of your choice.

   k8s/cloud/public/proxy_envoy.yaml
   k8s/cloud/public/domain_config.yaml
   scripts/create_cloud_secrets.sh
   

6. Install mkcert following the directions here. Pixie uses SSL to securely communicate between Pixie Cloud and the UI. Self-managed Pixie Cloud requires managing your own certificates. mkcert is a simple tool to create and install a local certificate authority (CA) in the system root store in order to generate locally-trusted certificates.

7. Start mkcert. This command will set up local CA and create a root certificate that Chrome and your CLI will now trust. To access Pixie Cloud from different machine that the one it was set up on, you will need to install this certificate there as well.

   mkcert -install
   

8. Create the plc namespace. This namespace is not currently configurable. Several of the install scripts expect Pixie Cloud to be deployed to the plc namespace.

   kubectl create namespace plc
   

9. Create the Pixie Cloud secrets. From the top level pixie/ directory, run:

   ./scripts/create_cloud_secrets.sh
   

10. Install kustomize following the directions here.

11. Deploy Pixie Cloud dependencies and wait for all pods within the plc namespace to become ready and available before proceeding to the next step. If there is an error, you may need to retry this step.

    kustomize build k8s/cloud_deps/base/elastic/operator | kubectl apply -f -
    kustomize build k8s/cloud_deps/public | kubectl apply -f -
    

12. Deploy Pixie Cloud.

    kustomize build k8s/cloud/public/ | kubectl apply -f -
    

13. Wait for all pods within the plc namespace to become ready and available. Note that you may have one or more create-hydra-client-job pod errors, but as long as long as another instance of that pod successfully completes, that is ok.

    kubectl get pods -n plc
    

1. Ensure that the cloud-proxy-service and vzconn-service LoadBalancer services have External IPs assigned. If you are running Pixie Cloud on minikube, you likely need to run minikube tunnel before continuing with this setup.

   minikube tunnel # if running on minikube
   kubectl get service cloud-proxy-service -n plc
   kubectl get service vzconn-service -n plc
   

2. Setup your DNS. This produces a dev_dns_updater binary in the top level pixie directory.

   go build src/utils/dev_dns_updater/dev_dns_updater.go
   

3. You'll need to hardcode in your kube config. If you are using a custom domain name, specify it as the value of the --domain-name flag. Leave this tab open.

   ./dev_dns_updater --domain-name="dev.withpixie.dev"  --kubeconfig=$HOME/.kube/config --n=plc
   

4. Navigate to dev.withpixie.dev in your browser. Make sure that the network you are on can access your cluster.

1. Login to the admin account using admin@default.com for the email and admin for the password.

If you would like to change the admin account credentials, modify the email and password values for the ADMIN_IDENTITY in k8s/cloud/base/ory_auth/kratos/kratos_deployment.yaml. You will need to redeploy from scratch if you've already deployed with the default admin account, since the admin account is automatically provisioned for you.

Add users to your organization to share access to Pixie Live Views, query running clusters, and deploy new Pixie clusters. For instructions, see the User Management & Sharing reference docs.

1. Set the cloud address with an environment variable. If you configured a custom domain name, use that as the variable's value:

   export PL_CLOUD_ADDR=dev.withpixie.dev
   

2. Install Pixie's CLI. The easiest way to install Pixie's CLI is using the install script:

   # Copy and run command to install the Pixie CLI.
   bash -c "$(curl -fsSL https://withpixie.ai/install.sh)"
   

   For alternate install options (Docker, Debian package, RPM, direct download of the binary) see the CLI Install page.

Pixie's CLI is the fastest and easiest way to deploy Pixie. You can also deploy Pixie using YAML or Helm.

To deploy Pixie using the CLI:

# Deploy the Pixie Platform in your K8s cluster (No OLM present on cluster).
px deploy --dev_cloud_namespace plc
# Deploy the Pixie Platform in your K8s cluster (OLM already exists on cluster).
px deploy  --dev_cloud_namespace plc --deploy_olm=false
# Deploy Pixie with a specific memory limit (2Gi is the default, 1Gi is the minimum recommended)
px deploy --dev_cloud_namespace plc --pem_memory_limit=1Gi

Pixie will deploy pods to the pl, plc, px-operator, and olm(if deploying the OLM) namespaces.

For more deploy options that you can specify to configure Pixie, refer to our deploy options.

Check out the next section of our docs for Using Pixie. You can also check out our Tutorials section.

Learn how to use Pixie for

• Network Monitoring
• Infra Health
• Service Performance
• Database Query Profiling
• Request Tracing
• Kafka Monitoring

There are two options for deploying Pixie to another Kubernetes cluster.

This will spin up a separate instance of Pixie Cloud for each Pixie deployment that you have.

If you select this option, each of your Pixie deployments will point to the same instance of Pixie Cloud. In order to ensure that all of your clusters can access Pixie Cloud, you will need to do the following:

• Rename your Pixie Cloud address from dev.withpixie.dev to something specific to your environment.
• Set up DNS rules for your new Pixie Cloud address. The specifics of this will depend on your environment.
• Ensure your cloud TLS certificates are for your new Pixie Cloud domain.
• Ping the new Pixie Cloud address from the new cluster before deploying Pixie to make sure traffic is successfully reaching Pixie Cloud.

Please refer to the Production Readiness guide for detailed instructions.

Please see our Troubleshooting guide, reach out on our Community Slack or file an issue on GitHub.

Community Cloud for PixieAir Gapped Pixie