API 가이드

Client

xinference.client.Client(base_url[, api_key])
xinference.client.Client.describe_model(...)	Get model information via RESTful APIs.
xinference.client.Client.get_model(model_uid)	Launch the model based on the parameters on the server via RESTful APIs.
xinference.client.Client.get_model_registration(...)	Get the model with the model type and model name registered on the server.
xinference.client.Client.get_launch_model_progress(...)	Get progress of the specific model.
xinference.client.Client.cancel_launch_model(...)	Cancel launching model.
xinference.client.Client.get_instance_info(...)
xinference.client.Client.launch_model(model_name)	Launch the model based on the parameters on the server via RESTful APIs.
xinference.client.Client.list_model_registrations(...)	List models registered on the server.
xinference.client.Client.list_models()	Retrieve the model specifications from the Server.
xinference.client.Client.list_cached_models([...])	Get a list of cached models.
xinference.client.Client.list_deletable_models(...)	Get the cached models with the model path cached on the server.
xinference.client.Client.confirm_and_remove_model(...)	Remove the cached models with the model name cached on the server.
xinference.client.Client.query_engine_by_model_name(...)	Get the engine parameters with the model name registered on the server.
xinference.client.Client.register_model(...)	Register a custom model.
xinference.client.Client.terminate_model(...)	Terminate the specific model running on the server.
xinference.client.Client.abort_request(...)	Abort a request.
xinference.client.Client.vllm_models()
xinference.client.Client.login(username, ...)
xinference.client.Client.get_workers_info()
xinference.client.Client.get_supervisor_info()
xinference.client.Client.get_progress(request_id)
xinference.client.Client.abort_cluster()
xinference.client.Client.unregister_model(...)	Unregister a custom model.

Model Handles

ChatModelHandle

xinference.client.handlers.ChatModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulChatModelHandle`의 별칭
xinference.client.handlers.ChatModelHandle.chat(...)	Given a list of messages comprising a conversation, the model will return a response via RESTful APIs.
xinference.client.handlers.ChatModelHandle.generate(prompt)	Creates a completion for the provided prompt and parameters via RESTful APIs.

EmbeddingModelHandle

xinference.client.handlers.EmbeddingModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulEmbeddingModelHandle`의 별칭
xinference.client.handlers.EmbeddingModelHandle.create_embedding(...)	Create an Embedding from user input via RESTful APIs.

RerankModelHandle

xinference.client.restful.restful_client.RESTfulRerankModelHandle(...)
xinference.client.restful.restful_client.RESTfulRerankModelHandle.rerank(...)	Returns an ordered list of documents ordered by their relevance to the provided query.

GenerateModelHandle

xinference.client.handlers.GenerateModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulGenerateModelHandle`의 별칭
xinference.client.handlers.GenerateModelHandle.generate(prompt)	Creates a completion for the provided prompt and parameters via RESTful APIs.

ImageModelHandle

xinference.client.handlers.ImageModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulImageModelHandle`의 별칭
xinference.client.handlers.ImageModelHandle.text_to_image(prompt)	Creates an image by the input text.

AudioModelHandle

xinference.client.handlers.AudioModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulAudioModelHandle`의 별칭
xinference.client.handlers.AudioModelHandle.transcriptions(audio)	Transcribes audio into the input language.
xinference.client.handlers.AudioModelHandle.translations(audio)	Translates audio into English.
xinference.client.handlers.AudioModelHandle.speech(input)	Generates audio from the input text.

FlexibleModelHandle

xinference.client.restful.restful_client.RESTfulFlexibleModelHandle(...)
xinference.client.restful.restful_client.RESTfulFlexibleModelHandle.infer(...)	Call flexible model.

VideoModelHandle

xinference.client.handlers.VideoModelHandle	:py:class:`~xinference.client.restful.restful_client.RESTfulVideoModelHandle`의 별칭
xinference.client.handlers.VideoModelHandle.text_to_video(prompt)	Creates a video by the input text.