neildlf commited on
Commit
b84e6f0
·
verified ·
1 Parent(s): 44be29c

Eguneratuta, bukatutzat eman genezake, emaitzak falta dira soilik

Browse files
Files changed (1) hide show
  1. README.md +93 -3
README.md CHANGED
@@ -45,9 +45,11 @@ tags:
45
 
46
  This model achieves state-of-the-art performance on zero-shot Named Entity Recognition (NER) by first training on `GuideX`, a large-scale synthetic dataset with executable guidelines, and then fine-tuning on a collection of gold-standard IE datasets.
47
 
48
- - **Homepage:** [https://neilus03.github.io/guidex.com/](https://neilus03.github.io/guidex.com/)
49
- - **Paper:** [GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction](https://arxiv.org/abs/2506.00649)
50
- - **Code & Data:** The code and data for reproducing the GuideX methodology are available on the project homepage.
 
 
51
 
52
  ## Model Description
53
 
@@ -58,3 +60,91 @@ This model achieves state-of-the-art performance on zero-shot Named Entity Recog
58
  - **License:** Llama 2 Community License
59
  - **Finetuned from model:** `meta-llama/Llama-3.1-8B`
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
  This model achieves state-of-the-art performance on zero-shot Named Entity Recognition (NER) by first training on `GuideX`, a large-scale synthetic dataset with executable guidelines, and then fine-tuning on a collection of gold-standard IE datasets.
47
 
48
+ - 💻 **Project Page:** [https://neilus03.github.io/guidex.com/](https://neilus03.github.io/guidex.com/)
49
+ - 📒 **Code:** [GuideX codebase](https://github.com/Neilus03/GUIDEX)
50
+ - 📖 **Paper:** [GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction](https://arxiv.org/abs/2506.00649)
51
+ - 🐕 **GuideX Colection in the 🤗HuggingFace Hub:** [GuideX Collection](https://huggingface.co/collections/neildlf/guidex-6842ef478e8d9bb0a00c844d)
52
+ - 🚀 **Example Jupyter Notebooks:** [GuideX Notebooks](https://github.com/Neilus03/GUIDEX/tree/dev-neil/notebooks)
53
 
54
  ## Model Description
55
 
 
60
  - **License:** Llama 2 Community License
61
  - **Finetuned from model:** `meta-llama/Llama-3.1-8B`
62
 
63
+ ## Schema definition and inference example
64
+
65
+ The labels are represented as Python classes, and the guidelines or instructions are introduced as docstrings. The model start generating after the `result = [` line.
66
+ ```Python
67
+ # Entity definitions
68
+ @dataclass
69
+ class Launcher(Template):
70
+ """Refers to a vehicle designed primarily to transport payloads from the Earth's
71
+ surface to space. Launchers can carry various payloads, including satellites,
72
+ crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit.
73
+ They are usually multi-stage vehicles that use rocket engines for propulsion."""
74
+
75
+ mention: str
76
+ """
77
+ The name of the launcher vehicle.
78
+ Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5"
79
+ """
80
+ space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace"
81
+ crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin"
82
+
83
+
84
+ @dataclass
85
+ class Mission(Template):
86
+ """Any planned or accomplished journey beyond Earth's atmosphere with specific objectives,
87
+ either crewed or uncrewed. It includes missions to satellites, the International
88
+ Space Station (ISS), other celestial bodies, and deep space."""
89
+
90
+ mention: str
91
+ """
92
+ The name of the mission.
93
+ Such as: "Apollo 11", "Artemis", "Mercury"
94
+ """
95
+ date: str # The start date of the mission
96
+ departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana"
97
+ destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn"
98
+
99
+ # This is the text to analyze
100
+ text = (
101
+ "The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica,"
102
+ "carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez."
103
+ )
104
+
105
+ # The annotation instances that take place in the text above are listed here
106
+ result = [
107
+ Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'),
108
+ Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])
109
+ ]
110
+ ```
111
+
112
+ ## How to Get Started with the Model
113
+
114
+ Please read our [🚀 Example Jupyter Notebooks](https://github.com/Neilus03/GUIDEX/tree/dev-neil/notebooks) to get started with GuideX.
115
+
116
+ The best way to load the model is using our custom `load_model` fuction. However, you can also load them using the AutoModelForCausalLM class.
117
+
118
+ **Important**: Our flash attention implementation has small numerical differences compared to the attention implementation in Huggingface.
119
+ You must use the flag `trust_remote_code=True` or you will get inferior results. Flash attention requires an available CUDA GPU. Running GuideX
120
+ pre-trained models on a CPU is not supported. We plan to address this in future releases. First, install flash attention 2:
121
+ ```bash
122
+ pip install flash-attn --no-build-isolation
123
+ pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
124
+ ```
125
+
126
+ Then you can load the model using
127
+
128
+ ```python
129
+ import torch
130
+ from transformers import AutoTokenizer, AutoModelForCausalLM
131
+
132
+ tokenizer = AutoTokenizer.from_pretrained("HiTZ/Llama-3.1-GuideX-8B")
133
+ model = AutoModelForCausalLM.from_pretrained("HiTZ/Llama-3.1-GuideX-8B", trust_remote_code=True, torch_dtype=torch.bfloat16)
134
+ model.to("cuda")
135
+ ```
136
+
137
+ Read our [🚀 Example Jupyter Notebooks](https://github.com/hitz-zentroa/GoLLIE/tree/main/notebooks) to learn how to easily define guidelines, generate model inputs and parse the output!
138
+
139
+ ```
140
+ ## Citation
141
+ @misc{delafuente2025guidexguidedsyntheticdata,
142
+ title={GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction},
143
+ author={Neil De La Fuente and Oscar Sainz and Iker García-Ferrero and Eneko Agirre},
144
+ year={2025},
145
+ eprint={2506.00649},
146
+ archivePrefix={arXiv},
147
+ primaryClass={cs.CL},
148
+ url={https://arxiv.org/abs/2506.00649},
149
+ }
150
+ ```