Chunhua Liao commited on
Commit
29a4f30
·
1 Parent(s): 17b9e8a

add a config.yaml file

Browse files
README.md CHANGED
@@ -46,7 +46,7 @@ The `proposal-gen-v1.py` script implements a multi-agent system that iteratively
46
  uvicorn proposal-gen-v1:app --host 0.0.0.0 --port 8000
47
  ```
48
  4. **Access the Web Interface:**
49
- Open a web browser and go to `http://localhost:8000`.
50
  5. **Enter Research Goal:**
51
  Enter your research goal in the text area provided.
52
  6. **Submit and Run:**
@@ -66,6 +66,28 @@ The system will generate a list of hypotheses related to the research goal. Each
66
 
67
  The web interface will display the top-ranked hypotheses after each cycle, along with a meta-review critique and suggested next steps. The results are iterative, meaning that the hypotheses should improve over multiple cycles. Log files are created in the `results/` directory for each run.
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  ## Known Limitations
70
 
71
  * **LLM Dependency:** The quality of the results heavily depends on the capabilities of the underlying LLM.
 
46
  uvicorn proposal-gen-v1:app --host 0.0.0.0 --port 8000
47
  ```
48
  4. **Access the Web Interface:**
49
+ Open a web browser and go to `http://localhost:8000`. (Note: The server log may show `http://0.0.0.0:8000`, which means the server is listening on all network interfaces. However, you should use `localhost` in your browser to access the server from your local machine. You cannot directly type `0.0.0.0` into your browser's address bar.)
50
  5. **Enter Research Goal:**
51
  Enter your research goal in the text area provided.
52
  6. **Submit and Run:**
 
66
 
67
  The web interface will display the top-ranked hypotheses after each cycle, along with a meta-review critique and suggested next steps. The results are iterative, meaning that the hypotheses should improve over multiple cycles. Log files are created in the `results/` directory for each run.
68
 
69
+ ## Configuration (config.yaml)
70
+
71
+ The `config.yaml` file contains settings that control the behavior of the AI Co-Scientist system. Here's a detailed explanation of each option:
72
+
73
+ * **`openrouter_base_url`**: This specifies the base URL for the OpenRouter API. OpenRouter acts as a proxy to various LLMs, providing a consistent interface. The default value is `"https://openrouter.ai/api/v1"`, which should work without modification.
74
+
75
+ * **`llm_model`**: This setting determines which Large Language Model (LLM) the system will use. The default is `"google/gemini-2.0-flash-thinking-exp:free"`, which is a free model from Google, hosted on OpenRouter. You can change this to use a different model, but ensure it's compatible with the OpenRouter API and the system's prompts. Refer to the OpenRouter documentation for available models and their identifiers.
76
+
77
+ * **`num_hypotheses`**: This controls the number of initial hypotheses generated in each cycle. The default value is `3`. Increasing this number will explore a broader range of ideas, but may also increase processing time and API costs (if using a paid LLM).
78
+
79
+ * **`elo_k_factor`**: This parameter, used in the Elo rating system, determines how much the Elo scores change after each comparison between hypotheses. A higher `elo_k_factor` (default is `32`) means that scores will change more dramatically, making the ranking more sensitive to individual comparisons. A lower value will result in slower, more gradual changes in ranking.
80
+
81
+ * **`top_k_hypotheses`**: This setting specifies how many of the top-ranked hypotheses are used by the `EvolutionAgent` to create new hypotheses. The default is `2`. Increasing this value might lead to more diverse combinations, but could also dilute the influence of the very best hypotheses.
82
+
83
+ * **`logging_level`**: This controls the verbosity of the logging output. Valid values are `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, and `"CRITICAL"`. The default is `"INFO"`. `"DEBUG"` provides the most detailed information, while `"CRITICAL"` only logs the most severe errors.
84
+
85
+ * **`log_file_name`**: This is the base name for the log files (without the extension). Log files are stored in the `results/` directory. The default is `"app"`. The system automatically adds a timestamp and the `.txt` extension to the log file name (e.g., `app_2025-02-28_09-18-00.txt`).
86
+
87
+ * **`fastapi_host`**: This setting controls the network interface that the FastAPI application will listen on. The default value, `"0.0.0.0"`, makes the application accessible from any network interface, including your local machine and potentially other computers on your network. You could change this to `"127.0.0.1"` to restrict access to only your local machine.
88
+
89
+ * **`fastapi_port`**: This specifies the port number that the FastAPI application will use. The default is `8000`. You can change this if you have another application already using port 8000 or if you prefer a different port for other reasons.
90
+
91
  ## Known Limitations
92
 
93
  * **LLM Dependency:** The quality of the results heavily depends on the capabilities of the underlying LLM.
app_log_2025-02-28_09-27-02.txt ADDED
The diff for this file is too large to render. See raw diff
 
proposal-gen-v1.py CHANGED
@@ -8,8 +8,10 @@ from openai import OpenAI
8
  import os
9
  import datetime
10
  from fastapi import FastAPI, HTTPException, responses
 
11
  from pydantic import BaseModel
12
  import uvicorn
 
13
 
14
  # Configure logging for production readiness.
15
  # logging.basicConfig(
@@ -19,20 +21,42 @@ import uvicorn
19
  # )
20
  # logger = logging.getLogger("co_scientist") # global logger
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  def setup_logger(log_filename):
23
  logger = logging.getLogger(log_filename) # Create a logger with the filename
24
- logger.setLevel(logging.INFO)
25
  formatter = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s")
26
 
27
  # Remove existing handlers to avoid duplicate logs
28
  for handler in logger.handlers[:]:
29
  logger.removeHandler(handler)
30
 
31
- file_handler = logging.FileHandler(log_filename)
32
  file_handler.setFormatter(formatter)
33
  logger.addHandler(file_handler)
34
  return logger
35
 
 
 
 
36
  def call_llm(prompt: str) -> str:
37
  """
38
  Calls an LLM via the OpenRouter API and returns the response.
@@ -44,16 +68,14 @@ def call_llm(prompt: str) -> str:
44
  str: The LLM's response.
45
  """
46
  client = OpenAI(
47
- base_url="https://openrouter.ai/api/v1",
48
- api_key=os.getenv("OPENROUTER_API_KEY")
49
  )
50
 
51
  try:
52
  completion = client.chat.completions.create(
53
- model="google/gemini-2.0-flash-thinking-exp:free", # Or any other suitable model
54
- messages=[
55
- {"role": "user", "content": prompt}
56
- ],
57
  )
58
  except Exception as e:
59
  # If the library raises an exception (e.g., for invalid key, rate limit, etc.)
@@ -281,7 +303,7 @@ def run_pairwise_debate(hypoA: Hypothesis, hypoB: Hypothesis) -> Hypothesis:
281
  hypoA.hypothesis_id, scoreA, hypoB.hypothesis_id, scoreB, winner.hypothesis_id)
282
  return winner
283
 
284
- def update_elo(winner: Hypothesis, loser: Hypothesis, k_factor: int = 32):
285
  """
286
  Updates the Elo scores of two hypotheses after a pairwise comparison.
287
 
@@ -355,9 +377,9 @@ class GenerationAgent:
355
  prompt = (
356
  f"Research Goal: {research_goal.description}\n"
357
  f"Constraints: {research_goal.constraints}\n"
358
- "Please propose 2 new hypotheses with rationale.\n"
359
  )
360
- raw_output = call_llm_for_generation(prompt, num_hypotheses=2)
361
  new_hypos = []
362
  for idea in raw_output:
363
  hypo_id = generate_unique_id("G")
@@ -431,7 +453,7 @@ class EvolutionAgent:
431
  """
432
  active = context.get_active_hypotheses()
433
  sorted_by_elo = sorted(active, key=lambda h: h.elo_score, reverse=True)
434
- top_candidates = sorted_by_elo[:top_k]
435
  new_hypotheses = []
436
  if len(top_candidates) >= 2:
437
  new_h = combine_hypotheses(top_candidates[0], top_candidates[1])
@@ -567,6 +589,8 @@ global_context = ContextMemory()
567
  supervisor = SupervisorAgent()
568
  current_research_goal: Optional[ResearchGoal] = None
569
 
 
 
570
  @app.post("/research_goal", response_model=dict)
571
  def set_research_goal(goal: ResearchGoalRequest):
572
  """
@@ -714,4 +738,4 @@ async def root():
714
 
715
  if __name__ == "__main__":
716
  # Run with: uvicorn this_script:app --host 0.0.0.0 --port 8000
717
- uvicorn.run("proposal-gen-v1:app", host="0.0.0.0", port=8000, reload=False)
 
8
  import os
9
  import datetime
10
  from fastapi import FastAPI, HTTPException, responses
11
+ from fastapi.staticfiles import StaticFiles
12
  from pydantic import BaseModel
13
  import uvicorn
14
+ import yaml
15
 
16
  # Configure logging for production readiness.
17
  # logging.basicConfig(
 
21
  # )
22
  # logger = logging.getLogger("co_scientist") # global logger
23
 
24
+ def load_config(config_path: str) -> Dict:
25
+ """Loads the configuration from the specified YAML file."""
26
+ try:
27
+ with open(config_path, "r") as f:
28
+ config = yaml.safe_load(f)
29
+ # Convert logging level string to actual level
30
+ config["logging_level"] = getattr(logging, config["logging_level"].upper(), logging.INFO)
31
+ return config
32
+ except FileNotFoundError:
33
+ print(f"Error: Configuration file not found at {config_path}")
34
+ exit(1)
35
+ except yaml.YAMLError as e:
36
+ print(f"Error parsing YAML in {config_path}: {e}")
37
+ exit(1)
38
+ except AttributeError as e:
39
+ print("Error: Invalid logging level in config file")
40
+ exit(1)
41
+
42
+
43
  def setup_logger(log_filename):
44
  logger = logging.getLogger(log_filename) # Create a logger with the filename
45
+ logger.setLevel(config["logging_level"])
46
  formatter = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s")
47
 
48
  # Remove existing handlers to avoid duplicate logs
49
  for handler in logger.handlers[:]:
50
  logger.removeHandler(handler)
51
 
52
+ file_handler = logging.FileHandler(f"{config['log_file_name']}_{log_filename}")
53
  file_handler.setFormatter(formatter)
54
  logger.addHandler(file_handler)
55
  return logger
56
 
57
+ # Load configuration at the start
58
+ config = load_config("config.yaml")
59
+
60
  def call_llm(prompt: str) -> str:
61
  """
62
  Calls an LLM via the OpenRouter API and returns the response.
 
68
  str: The LLM's response.
69
  """
70
  client = OpenAI(
71
+ base_url=config["openrouter_base_url"],
72
+ api_key=os.getenv("OPENROUTER_API_KEY"),
73
  )
74
 
75
  try:
76
  completion = client.chat.completions.create(
77
+ model=config["llm_model"],
78
+ messages=[{"role": "user", "content": prompt}],
 
 
79
  )
80
  except Exception as e:
81
  # If the library raises an exception (e.g., for invalid key, rate limit, etc.)
 
303
  hypoA.hypothesis_id, scoreA, hypoB.hypothesis_id, scoreB, winner.hypothesis_id)
304
  return winner
305
 
306
+ def update_elo(winner: Hypothesis, loser: Hypothesis, k_factor: int = config["elo_k_factor"]):
307
  """
308
  Updates the Elo scores of two hypotheses after a pairwise comparison.
309
 
 
377
  prompt = (
378
  f"Research Goal: {research_goal.description}\n"
379
  f"Constraints: {research_goal.constraints}\n"
380
+ f"Please propose {config['num_hypotheses']} new hypotheses with rationale.\n"
381
  )
382
+ raw_output = call_llm_for_generation(prompt, num_hypotheses=config["num_hypotheses"])
383
  new_hypos = []
384
  for idea in raw_output:
385
  hypo_id = generate_unique_id("G")
 
453
  """
454
  active = context.get_active_hypotheses()
455
  sorted_by_elo = sorted(active, key=lambda h: h.elo_score, reverse=True)
456
+ top_candidates = sorted_by_elo[:config["top_k_hypotheses"]]
457
  new_hypotheses = []
458
  if len(top_candidates) >= 2:
459
  new_h = combine_hypotheses(top_candidates[0], top_candidates[1])
 
589
  supervisor = SupervisorAgent()
590
  current_research_goal: Optional[ResearchGoal] = None
591
 
592
+ app.mount("/static", StaticFiles(directory="static"), name="static")
593
+
594
  @app.post("/research_goal", response_model=dict)
595
  def set_research_goal(goal: ResearchGoalRequest):
596
  """
 
738
 
739
  if __name__ == "__main__":
740
  # Run with: uvicorn this_script:app --host 0.0.0.0 --port 8000
741
+ uvicorn.run("proposal-gen-v1:app", host=config["fastapi_host"], port=config["fastapi_port"], reload=False)
requirements.txt CHANGED
@@ -1,4 +1,5 @@
1
  openai
2
  fastapi
3
- pydantic
4
  uvicorn
 
 
 
1
  openai
2
  fastapi
 
3
  uvicorn
4
+ pydantic
5
+ PyYAML