Spaces:
Sleeping
Sleeping
Chunhua Liao
commited on
Commit
·
29a4f30
1
Parent(s):
17b9e8a
add a config.yaml file
Browse files- README.md +23 -1
- app_log_2025-02-28_09-27-02.txt +0 -0
- proposal-gen-v1.py +37 -13
- requirements.txt +2 -1
README.md
CHANGED
@@ -46,7 +46,7 @@ The `proposal-gen-v1.py` script implements a multi-agent system that iteratively
|
|
46 |
uvicorn proposal-gen-v1:app --host 0.0.0.0 --port 8000
|
47 |
```
|
48 |
4. **Access the Web Interface:**
|
49 |
-
Open a web browser and go to `http://localhost:8000`.
|
50 |
5. **Enter Research Goal:**
|
51 |
Enter your research goal in the text area provided.
|
52 |
6. **Submit and Run:**
|
@@ -66,6 +66,28 @@ The system will generate a list of hypotheses related to the research goal. Each
|
|
66 |
|
67 |
The web interface will display the top-ranked hypotheses after each cycle, along with a meta-review critique and suggested next steps. The results are iterative, meaning that the hypotheses should improve over multiple cycles. Log files are created in the `results/` directory for each run.
|
68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
69 |
## Known Limitations
|
70 |
|
71 |
* **LLM Dependency:** The quality of the results heavily depends on the capabilities of the underlying LLM.
|
|
|
46 |
uvicorn proposal-gen-v1:app --host 0.0.0.0 --port 8000
|
47 |
```
|
48 |
4. **Access the Web Interface:**
|
49 |
+
Open a web browser and go to `http://localhost:8000`. (Note: The server log may show `http://0.0.0.0:8000`, which means the server is listening on all network interfaces. However, you should use `localhost` in your browser to access the server from your local machine. You cannot directly type `0.0.0.0` into your browser's address bar.)
|
50 |
5. **Enter Research Goal:**
|
51 |
Enter your research goal in the text area provided.
|
52 |
6. **Submit and Run:**
|
|
|
66 |
|
67 |
The web interface will display the top-ranked hypotheses after each cycle, along with a meta-review critique and suggested next steps. The results are iterative, meaning that the hypotheses should improve over multiple cycles. Log files are created in the `results/` directory for each run.
|
68 |
|
69 |
+
## Configuration (config.yaml)
|
70 |
+
|
71 |
+
The `config.yaml` file contains settings that control the behavior of the AI Co-Scientist system. Here's a detailed explanation of each option:
|
72 |
+
|
73 |
+
* **`openrouter_base_url`**: This specifies the base URL for the OpenRouter API. OpenRouter acts as a proxy to various LLMs, providing a consistent interface. The default value is `"https://openrouter.ai/api/v1"`, which should work without modification.
|
74 |
+
|
75 |
+
* **`llm_model`**: This setting determines which Large Language Model (LLM) the system will use. The default is `"google/gemini-2.0-flash-thinking-exp:free"`, which is a free model from Google, hosted on OpenRouter. You can change this to use a different model, but ensure it's compatible with the OpenRouter API and the system's prompts. Refer to the OpenRouter documentation for available models and their identifiers.
|
76 |
+
|
77 |
+
* **`num_hypotheses`**: This controls the number of initial hypotheses generated in each cycle. The default value is `3`. Increasing this number will explore a broader range of ideas, but may also increase processing time and API costs (if using a paid LLM).
|
78 |
+
|
79 |
+
* **`elo_k_factor`**: This parameter, used in the Elo rating system, determines how much the Elo scores change after each comparison between hypotheses. A higher `elo_k_factor` (default is `32`) means that scores will change more dramatically, making the ranking more sensitive to individual comparisons. A lower value will result in slower, more gradual changes in ranking.
|
80 |
+
|
81 |
+
* **`top_k_hypotheses`**: This setting specifies how many of the top-ranked hypotheses are used by the `EvolutionAgent` to create new hypotheses. The default is `2`. Increasing this value might lead to more diverse combinations, but could also dilute the influence of the very best hypotheses.
|
82 |
+
|
83 |
+
* **`logging_level`**: This controls the verbosity of the logging output. Valid values are `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, and `"CRITICAL"`. The default is `"INFO"`. `"DEBUG"` provides the most detailed information, while `"CRITICAL"` only logs the most severe errors.
|
84 |
+
|
85 |
+
* **`log_file_name`**: This is the base name for the log files (without the extension). Log files are stored in the `results/` directory. The default is `"app"`. The system automatically adds a timestamp and the `.txt` extension to the log file name (e.g., `app_2025-02-28_09-18-00.txt`).
|
86 |
+
|
87 |
+
* **`fastapi_host`**: This setting controls the network interface that the FastAPI application will listen on. The default value, `"0.0.0.0"`, makes the application accessible from any network interface, including your local machine and potentially other computers on your network. You could change this to `"127.0.0.1"` to restrict access to only your local machine.
|
88 |
+
|
89 |
+
* **`fastapi_port`**: This specifies the port number that the FastAPI application will use. The default is `8000`. You can change this if you have another application already using port 8000 or if you prefer a different port for other reasons.
|
90 |
+
|
91 |
## Known Limitations
|
92 |
|
93 |
* **LLM Dependency:** The quality of the results heavily depends on the capabilities of the underlying LLM.
|
app_log_2025-02-28_09-27-02.txt
ADDED
The diff for this file is too large to render.
See raw diff
|
|
proposal-gen-v1.py
CHANGED
@@ -8,8 +8,10 @@ from openai import OpenAI
|
|
8 |
import os
|
9 |
import datetime
|
10 |
from fastapi import FastAPI, HTTPException, responses
|
|
|
11 |
from pydantic import BaseModel
|
12 |
import uvicorn
|
|
|
13 |
|
14 |
# Configure logging for production readiness.
|
15 |
# logging.basicConfig(
|
@@ -19,20 +21,42 @@ import uvicorn
|
|
19 |
# )
|
20 |
# logger = logging.getLogger("co_scientist") # global logger
|
21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
def setup_logger(log_filename):
|
23 |
logger = logging.getLogger(log_filename) # Create a logger with the filename
|
24 |
-
logger.setLevel(
|
25 |
formatter = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s")
|
26 |
|
27 |
# Remove existing handlers to avoid duplicate logs
|
28 |
for handler in logger.handlers[:]:
|
29 |
logger.removeHandler(handler)
|
30 |
|
31 |
-
file_handler = logging.FileHandler(log_filename)
|
32 |
file_handler.setFormatter(formatter)
|
33 |
logger.addHandler(file_handler)
|
34 |
return logger
|
35 |
|
|
|
|
|
|
|
36 |
def call_llm(prompt: str) -> str:
|
37 |
"""
|
38 |
Calls an LLM via the OpenRouter API and returns the response.
|
@@ -44,16 +68,14 @@ def call_llm(prompt: str) -> str:
|
|
44 |
str: The LLM's response.
|
45 |
"""
|
46 |
client = OpenAI(
|
47 |
-
|
48 |
-
|
49 |
)
|
50 |
|
51 |
try:
|
52 |
completion = client.chat.completions.create(
|
53 |
-
model="
|
54 |
-
messages=[
|
55 |
-
{"role": "user", "content": prompt}
|
56 |
-
],
|
57 |
)
|
58 |
except Exception as e:
|
59 |
# If the library raises an exception (e.g., for invalid key, rate limit, etc.)
|
@@ -281,7 +303,7 @@ def run_pairwise_debate(hypoA: Hypothesis, hypoB: Hypothesis) -> Hypothesis:
|
|
281 |
hypoA.hypothesis_id, scoreA, hypoB.hypothesis_id, scoreB, winner.hypothesis_id)
|
282 |
return winner
|
283 |
|
284 |
-
def update_elo(winner: Hypothesis, loser: Hypothesis, k_factor: int =
|
285 |
"""
|
286 |
Updates the Elo scores of two hypotheses after a pairwise comparison.
|
287 |
|
@@ -355,9 +377,9 @@ class GenerationAgent:
|
|
355 |
prompt = (
|
356 |
f"Research Goal: {research_goal.description}\n"
|
357 |
f"Constraints: {research_goal.constraints}\n"
|
358 |
-
"Please propose
|
359 |
)
|
360 |
-
raw_output = call_llm_for_generation(prompt, num_hypotheses=
|
361 |
new_hypos = []
|
362 |
for idea in raw_output:
|
363 |
hypo_id = generate_unique_id("G")
|
@@ -431,7 +453,7 @@ class EvolutionAgent:
|
|
431 |
"""
|
432 |
active = context.get_active_hypotheses()
|
433 |
sorted_by_elo = sorted(active, key=lambda h: h.elo_score, reverse=True)
|
434 |
-
top_candidates = sorted_by_elo[:
|
435 |
new_hypotheses = []
|
436 |
if len(top_candidates) >= 2:
|
437 |
new_h = combine_hypotheses(top_candidates[0], top_candidates[1])
|
@@ -567,6 +589,8 @@ global_context = ContextMemory()
|
|
567 |
supervisor = SupervisorAgent()
|
568 |
current_research_goal: Optional[ResearchGoal] = None
|
569 |
|
|
|
|
|
570 |
@app.post("/research_goal", response_model=dict)
|
571 |
def set_research_goal(goal: ResearchGoalRequest):
|
572 |
"""
|
@@ -714,4 +738,4 @@ async def root():
|
|
714 |
|
715 |
if __name__ == "__main__":
|
716 |
# Run with: uvicorn this_script:app --host 0.0.0.0 --port 8000
|
717 |
-
uvicorn.run("proposal-gen-v1:app", host="
|
|
|
8 |
import os
|
9 |
import datetime
|
10 |
from fastapi import FastAPI, HTTPException, responses
|
11 |
+
from fastapi.staticfiles import StaticFiles
|
12 |
from pydantic import BaseModel
|
13 |
import uvicorn
|
14 |
+
import yaml
|
15 |
|
16 |
# Configure logging for production readiness.
|
17 |
# logging.basicConfig(
|
|
|
21 |
# )
|
22 |
# logger = logging.getLogger("co_scientist") # global logger
|
23 |
|
24 |
+
def load_config(config_path: str) -> Dict:
|
25 |
+
"""Loads the configuration from the specified YAML file."""
|
26 |
+
try:
|
27 |
+
with open(config_path, "r") as f:
|
28 |
+
config = yaml.safe_load(f)
|
29 |
+
# Convert logging level string to actual level
|
30 |
+
config["logging_level"] = getattr(logging, config["logging_level"].upper(), logging.INFO)
|
31 |
+
return config
|
32 |
+
except FileNotFoundError:
|
33 |
+
print(f"Error: Configuration file not found at {config_path}")
|
34 |
+
exit(1)
|
35 |
+
except yaml.YAMLError as e:
|
36 |
+
print(f"Error parsing YAML in {config_path}: {e}")
|
37 |
+
exit(1)
|
38 |
+
except AttributeError as e:
|
39 |
+
print("Error: Invalid logging level in config file")
|
40 |
+
exit(1)
|
41 |
+
|
42 |
+
|
43 |
def setup_logger(log_filename):
|
44 |
logger = logging.getLogger(log_filename) # Create a logger with the filename
|
45 |
+
logger.setLevel(config["logging_level"])
|
46 |
formatter = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s")
|
47 |
|
48 |
# Remove existing handlers to avoid duplicate logs
|
49 |
for handler in logger.handlers[:]:
|
50 |
logger.removeHandler(handler)
|
51 |
|
52 |
+
file_handler = logging.FileHandler(f"{config['log_file_name']}_{log_filename}")
|
53 |
file_handler.setFormatter(formatter)
|
54 |
logger.addHandler(file_handler)
|
55 |
return logger
|
56 |
|
57 |
+
# Load configuration at the start
|
58 |
+
config = load_config("config.yaml")
|
59 |
+
|
60 |
def call_llm(prompt: str) -> str:
|
61 |
"""
|
62 |
Calls an LLM via the OpenRouter API and returns the response.
|
|
|
68 |
str: The LLM's response.
|
69 |
"""
|
70 |
client = OpenAI(
|
71 |
+
base_url=config["openrouter_base_url"],
|
72 |
+
api_key=os.getenv("OPENROUTER_API_KEY"),
|
73 |
)
|
74 |
|
75 |
try:
|
76 |
completion = client.chat.completions.create(
|
77 |
+
model=config["llm_model"],
|
78 |
+
messages=[{"role": "user", "content": prompt}],
|
|
|
|
|
79 |
)
|
80 |
except Exception as e:
|
81 |
# If the library raises an exception (e.g., for invalid key, rate limit, etc.)
|
|
|
303 |
hypoA.hypothesis_id, scoreA, hypoB.hypothesis_id, scoreB, winner.hypothesis_id)
|
304 |
return winner
|
305 |
|
306 |
+
def update_elo(winner: Hypothesis, loser: Hypothesis, k_factor: int = config["elo_k_factor"]):
|
307 |
"""
|
308 |
Updates the Elo scores of two hypotheses after a pairwise comparison.
|
309 |
|
|
|
377 |
prompt = (
|
378 |
f"Research Goal: {research_goal.description}\n"
|
379 |
f"Constraints: {research_goal.constraints}\n"
|
380 |
+
f"Please propose {config['num_hypotheses']} new hypotheses with rationale.\n"
|
381 |
)
|
382 |
+
raw_output = call_llm_for_generation(prompt, num_hypotheses=config["num_hypotheses"])
|
383 |
new_hypos = []
|
384 |
for idea in raw_output:
|
385 |
hypo_id = generate_unique_id("G")
|
|
|
453 |
"""
|
454 |
active = context.get_active_hypotheses()
|
455 |
sorted_by_elo = sorted(active, key=lambda h: h.elo_score, reverse=True)
|
456 |
+
top_candidates = sorted_by_elo[:config["top_k_hypotheses"]]
|
457 |
new_hypotheses = []
|
458 |
if len(top_candidates) >= 2:
|
459 |
new_h = combine_hypotheses(top_candidates[0], top_candidates[1])
|
|
|
589 |
supervisor = SupervisorAgent()
|
590 |
current_research_goal: Optional[ResearchGoal] = None
|
591 |
|
592 |
+
app.mount("/static", StaticFiles(directory="static"), name="static")
|
593 |
+
|
594 |
@app.post("/research_goal", response_model=dict)
|
595 |
def set_research_goal(goal: ResearchGoalRequest):
|
596 |
"""
|
|
|
738 |
|
739 |
if __name__ == "__main__":
|
740 |
# Run with: uvicorn this_script:app --host 0.0.0.0 --port 8000
|
741 |
+
uvicorn.run("proposal-gen-v1:app", host=config["fastapi_host"], port=config["fastapi_port"], reload=False)
|
requirements.txt
CHANGED
@@ -1,4 +1,5 @@
|
|
1 |
openai
|
2 |
fastapi
|
3 |
-
pydantic
|
4 |
uvicorn
|
|
|
|
|
|
1 |
openai
|
2 |
fastapi
|
|
|
3 |
uvicorn
|
4 |
+
pydantic
|
5 |
+
PyYAML
|