Skip to main content

Anyscale

Anyscale is a platform to run, fine tune and scale LLMs via production-ready APIs. Anyscale Endpoints serve many open-source models in a cost-effective way.

Anyscale also provides an example how to setup LangChain with Anyscale for advanced chat agents.

Installation and Setup​

  • Get an Anyscale Service URL, route and API key and set them as environment variables (ANYSCALE_SERVICE_URL,ANYSCALE_SERVICE_ROUTE, ANYSCALE_SERVICE_TOKEN).
  • Please see the Anyscale docs for more details.

We have to install the openai package:

pip install openai

LLM​

See a usage example.

from langchain_community.llms.anyscale import Anyscale

API Reference:

Chat Models​

See a usage example.

from langchain_community.chat_models.anyscale import ChatAnyscale

API Reference:

Embeddings​

See a usage example.

from langchain_community.embeddings import AnyscaleEmbeddings

API Reference:

Help us out by providing feedback on this documentation page: