Ollama deploy LLM to use Graphrag,meets configuration problems #2117
Unanswered
Actioner1
asked this question in
Algorithm + Paper
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I use Ollama to deploy LLM,choose qwen3:0.6b as model ,embedding model choose quentinz/bge-large-zh-v1.5:latest
executing the following instruction to build the index of the file: python -m graphrag index --root ./graphrag_ollama
but it comes errors:
Failed to validate language model (default_chat_model) params litellm. InternalServerError: InternalServerError: OpenAIException - Connection error.
anyone have the same problem?
following is my settings_yaml
models:
default_chat_model:
type: chat
model_provider: openai
auth_type: api_key # or azure_managed_identity
#api_key: ${GRAPHRAG_API_KEY} # set this in the generated .env file, or remove if managed identity
api_key: ollama
model: qwen3:0.6b
api_base: https://localhost:11434/v1
encoding_model: cl100k_base
max_tokens: 2000
# api_version: 2024-05-01-preview
model_supports_json: true # recommended if this is available for your model.
#concurrent_requests: 25
concurrent_requests: 1
async_mode: threaded # or asyncio
retry_strategy: exponential_backoff
max_retries: 10
tokens_per_minute: null
requests_per_minute: null
default_embedding_model:
type: embedding
model_provider: openai
auth_type: api_key
#api_key: ${GRAPHRAG_API_KEY}
api_key: ollama
model: quentinz/bge-large-zh-v1.5:latest
encoding_model: cl100k_base
api_base: https://localhost:11434/api
# api_base: https://.openai.azure.com
# api_version: 2024-05-01-preview
#concurrent_requests: 25
concurrent_requests: 1
async_mode: threaded # or asyncio
retry_strategy: exponential_backoff
max_retries: 1
#max_retries: 10
tokens_per_minute: null
requests_per_minute: null
Input settings
input:
storage:
type: file # or blob
base_dir: "input"
file_type: text # [csv, text, json]
file_encoding: utf-8
chunks:
#size: 200
#overlap: 50
size: 300
overlap: 100
group_by_columns: [id]
Output/storage settings
If blob storage is specified in the following four sections,
connection_string and container_name must be provided
output:
type: file # [file, blob, cosmosdb]
base_dir: "output"
cache:
type: file # [file, blob, cosmosdb]
base_dir: "cache"
reporting:
type: file # [file, blob]
base_dir: "logs"
vector_store:
default_vector_store:
type: lancedb
db_uri: output\lancedb
container_name: default
Workflow settings
embed_text:
model_id: default_embedding_model
vector_store_id: default_vector_store
extract_graph:
model_id: default_chat_model
prompt: "prompts/extract_graph.txt"
entity_types: [organization,person,geo,event]
max_gleanings: 1
summarize_descriptions:
model_id: default_chat_model
prompt: "prompts/summarize_descriptions.txt"
max_length: 500
extract_graph_nlp:
text_analyzer:
extractor_type: regex_english # [regex_english, syntactic_parser, cfg]
async_mode: threaded # or asyncio
cluster_graph:
max_cluster_size: 10
extract_claims:
enabled: false
model_id: default_chat_model
prompt: "prompts/extract_claims.txt"
description: "Any claims or facts that could be relevant to information discovery."
max_gleanings: 1
community_reports:
model_id: default_chat_model
graph_prompt: "prompts/community_report_graph.txt"
text_prompt: "prompts/community_report_text.txt"
max_length: 2000
max_input_length: 8000
embed_graph:
enabled: false # if true, will generate node2vec embeddings for nodes
umap:
enabled: false # if true, will generate UMAP embeddings for nodes (embed_graph must also be enabled)
snapshots:
graphml: false
embeddings: false
Query settings
The prompt locations are required here, but each search method has a number of optional knobs that can be tuned.
See the config docs: https://microsoft.github.io/graphrag/config/yaml/#query
local_search:
chat_model_id: default_chat_model
embedding_model_id: default_embedding_model
prompt: "prompts/local_search_system_prompt.txt"
global_search:
chat_model_id: default_chat_model
map_prompt: "prompts/global_search_map_system_prompt.txt"
reduce_prompt: "prompts/global_search_reduce_system_prompt.txt"
knowledge_prompt: "prompts/global_search_knowledge_system_prompt.txt"
drift_search:
chat_model_id: default_chat_model
embedding_model_id: default_embedding_model
prompt: "prompts/drift_search_system_prompt.txt"
reduce_prompt: "prompts/drift_search_reduce_prompt.txt"
basic_search:
chat_model_id: default_chat_model
embedding_model_id: default_embedding_model
prompt: "prompts/basic_search_system_prompt.txt"
Beta Was this translation helpful? Give feedback.
All reactions