File size: 4,693 Bytes
04ccbd6 67fc22c 04ccbd6 6852591 0aaa86c 6852591 0aaa86c 6852591 e6c87e1 233c7b9 6852591 04ccbd6 67fc22c 04ccbd6 67fc22c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
---
datasets:
- neo4j/text2cypher-2024v1
base_model:
- google/gemma-2-9b-it
---
## Model Details
This is gguf format model for ```neo4j/text2cypher-gemma-2-9b-it-finetuned-2024v1```
### Model Description
This model serves as a demonstration of how fine-tuning foundational models using the Neo4j-Text2Cypher(2024) Dataset (https://huggingface.co/datasets/neo4j/text2cypher-2024v1) can enhance performance on the Text2Cypher task.
Please note, this is part of ongoing research and exploration, aimed at highlighting the dataset's potential rather than a production-ready solution.
Base model: google/gemma-2-9b-it
Dataset: neo4j/text2cypher-2024v1
An overview of the finetuned models and benchmarking results are shared at https://medium.com/p/d77be96ab65a and https://medium.com/p/b2203d1173b0
## Example Cypher generation
```python
import openai
# Define the instruction and helper functions
instruction = (
"Generate Cypher statement to query a graph database. "
"Use only the provided relationship types and properties in the schema. \n"
"Schema: {schema} \n Question: {question} \n Cypher output: "
)
def prepare_chat_prompt(question, schema):
# Build the messages list for the OpenAI API
return [
{
"role": "user",
"content": instruction.format(schema=schema, question=question),
}
]
def _postprocess_output_cypher(output_cypher: str) -> str:
# Remove any explanation text and code block markers
partition_by = "**Explanation:**"
output_cypher, _, _ = output_cypher.partition(partition_by)
output_cypher = output_cypher.strip("`\n")
output_cypher = output_cypher.lstrip("cypher\n")
output_cypher = output_cypher.strip("`\n ")
return output_cypher
# Configure the OpenAI API endpoint to your Ollama server.
# (Adjust the API base URL if your Ollama server is hosted at a different address/port.)
openai.api_base = "http://localhost:11434/v1"
openai.api_key = "YOUR_API_KEY" # Include if your setup requires an API key
# Set the model name as used by Ollama (this should match the name configured on your Ollama server)
model_name = "avinashm/text2cypher"
# Define the question and schema
question = "What are the movies of Tom Hanks?"
schema = "(:Actor)-[:ActedIn]->(:Movie)"
# Prepare the conversation messages
messages = prepare_chat_prompt(question=question, schema=schema)
# Call the API using similar generation parameters to your original script.
response = openai.ChatCompletion.create(
model=model_name,
messages=messages,
temperature=0.2,
max_tokens=512, # equivalent to max_new_tokens in your original script
top_p=0.9,
)
# Extract and post-process the output
raw_output = response["choices"][0]["message"]["content"]
output = _postprocess_output_cypher(raw_output)
print(output)
```
## NOTE: on creating your own schemas:
```
In the dataset we used, the schemas are already provided.
They are created either by Directly using the schema the input data source provided OR
Creating schema using neo4j-graphrag package (Check: SchemaReader.get_schema(...) function)
In your own Neo4j database, you can utilize neo4j-graphrag package::SchemaReader functions
```
# Example cypher queries to get schema:
```cypher
CALL apoc.meta.schema()
CALL db.schema.visualization()
```
## Bias, Risks, and Limitations
We need to be cautious about a few risks:
In our evaluation setup, the training and test sets come from the same data distribution (sampled from a larger dataset). If the data distribution changes, the results may not follow the same pattern.
The datasets used were gathered from publicly available sources. Over time, foundational models may access both the training and test sets, potentially achieving similar or even better results.
## Training Details
Training Procedure
Used RunPod with following setup:
```
1 x A100 PCIe
31 vCPU 117 GB RAM
runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04
On-Demand - Secure Cloud
60 GB Disk
60 GB Pod Volume
Training Hyperparameters
lora_config = LoraConfig( r=64, lora_alpha=64, target_modules=target_modules, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", )
sft_config = SFTConfig( dataset_text_field=dataset_text_field, per_device_train_batch_size=4, gradient_accumulation_steps=8, dataset_num_proc=16, max_seq_length=1600, logging_dir="./logs", num_train_epochs=1, learning_rate=2e-5, save_steps=5, save_total_limit=1, logging_steps=5, output_dir="outputs", optim="paged_adamw_8bit", save_strategy="steps", )
bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, )
```
|