Moved from https://huggingface.co/spaces/siyuwang541/ToDoAgent for the hackthon

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

LLM/Database/FalseNegative_samples.json

@@ -0,0 +1 @@
+[]

LLM/Database/Manual_filtering_samples.json

@@ -0,0 +1,190 @@
+ [ 
+{
+    "sender": "ASAP Sample",
+    "content": "王斯煜[表情]Vince 黑客松nv: @AlisaGG @LJK86 明天中午再对 WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331001504,
+    "user_id": 67619984,
+    "date": "2025-03-31T00:15:21",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]王斯煜[表情]Vince 黑客松nv: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002905,
+    "user_id": 67619984,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[4条]斯煜[表情]Vince: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002907,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[4条]斯煜[表情]Vince: 【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002908,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]斯煜[表情]Vince: 尊敬的*斯煜，您在我行办理的1笔个人贷款需于2024年12月08日17:00前还款，当期还款金额本息合计999999.84元，请您留意尾号0455的账户可用余额是否充足，避免因贷款逾期影响个人征信。具体贷款信息可通过工行网上银行、手机银行或致电贷款经办行查询。【工商银行】",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002909,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]斯煜[表情]Vince: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002910,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "王斯煜[表情]Vince 黑客松nv: WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://vc.feishu.cn/j/65644590...",
+  "app_name": "com.tencent.mm",
+  "message_id": 331115637,
+  "user_id": 67619984,
+  "date": "2025-03-31T11:56:36",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG陈G老师1010: 【驿收发】您的邮政包裹已到凯丰花园2栋驿站，请23:00前凭5-5-6530来取，详询18320926368",
+  "app_name": "com.tencent.mm",
+  "message_id": 331133944,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:39:49",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[2条]AlisaGG: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134011,
+  "user_id": 74797059,
+  "date": "2025-03-31T13:40:54"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[2条]AlisaGG陈G老师1010: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134045,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:40:54",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[3条]AlisaGG陈G老师1010: 取件通知\n取件码: 11724837\n运单号: 464285154986072\n取件地址: 深圳新一代产业园P2出入口内侧4号丰巢柜\n配送人员: 18124519013\n计费规则: 查看详情",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134346,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:43:17",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[4条]AlisaGG陈G老师1010: 取件再次提醒\n取件码：: 27696201\n配送公司：: 申通快递\n运单号：: 777293635831671\n配送���手机：: 13392809673\n取件地址：: 深圳新一代产业园P2出入口内侧4号丰巢柜",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134447,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:44:02",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[5条]李JK老师-1228: 【菜鸟驿站】请凭140-3-1019到菜鸟驿站取件，查询详情u.cainiao.com/53h4bSr7zrh",
+  "app_name": "com.tencent.mm",
+  "message_id": 331135751,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:57:28",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "刻刻: 【停机前提醒】尊敬的移动客户，您好！您的账户余额不足被限制使用。现提醒您需充值缴费至少32.49元，以确保您继续享受畅通的通信服务。诚邀您一键办理自动充服务，自动充值缴费更轻松：https://dx.10086.cn/7WyRLA。 心级服务、让爱连接【中国移动】",
+  "app_name": "com.tencent.mm",
+  "message_id": 331162269,
+  "user_id": 67619984,
+  "date": "2025-03-31T16:22:21",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[3条]LJK86: 您的号卡已在配送途中，物流单号SF3147624215612,点击：http://t.hn.189.cn/EZv6bazD，查询物流进度。如已签收请在有效期内先实名激活。客服热线4008155555【中国电信】",
+  "app_name": "com.tencent.mm",
+  "message_id": 330225703,
+  "user_id": 74797059,
+  "date": "2025-03-30T22:57:51",
+  "classification": "TruePositive"
+},
+{
+  "sender": "106875230196298038",
+  "content": "【京东快递】取件码C8042，您的快件尾号8980已送达成都东软学院配送营业点，地址：四川成都市都江堰市青城山镇东软大道1号东软学院电信营业厅旁 ，联系电话13032837084",
+  "app_name": "SMS",
+  "message_id": 328233028,
+  "user_id": 33642157,
+  "date": "2025-03-28T23:30:29",
+  "classification": "TruePositive"
+},
+{
+  "sender": "1068837016151",
+  "content": "【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。",
+  "app_name": "SMS",
+  "message_id": 322223721,
+  "user_id": 67619984,
+  "date": "2025-03-22T22:37:50",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Azure ToDoAgent",
+  "content": "陈格(alisagege.chen): WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://vc.feishu.cn/j/656445907\n\n手机拨号一键入会\n+862122504720,,656445907（中国大陆）\n4008208888,,656445907（中国大陆）\n\n根据所在地拨打号...",
+  "app_name": "com.ss.android.lark",
+  "message_id": 331150106,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:36",
+  "classification": "TruePositive"
+},
+{
+  "sender": "106917190196",
+  "content": "【广州自来水】(水费通知)尊敬的0001914043用户（大沙头三马路9号大院8号604）水量6（202503行度83），应缴水费11.88元，污水费5.7元。了解更多用水资讯可关注“广州自来水96968”微信公众号。",
+  "app_name": "SMS",
+  "message_id": 319155403,
+  "user_id": 67619984,
+  "date": "2025-03-19T15:54:16",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG陈G老师1010: 【招商银行】您本月办理分期还款可享限时2.3折优惠！打开掌上生活APP，搜“分期还款”立享折扣优惠！资格实时测评，拒收请回复R",
+  "app_name": "com.tencent.mm",
+  "message_id": 330231000,
+  "user_id": 67619984,
+  "date": "2025-03-30T23:10:59",
+  "classification": "TrueNegative"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[4条]AlisaGG: 【招商银行】您本月办理分期还款可享限时2.3折优惠！打开掌上生活APP，搜“分期还款”立享折扣优惠！资格实时测评，拒收请回复R",
+  "app_name": "com.tencent.mm",
+  "message_id": 330231005,
+  "user_id": 74797059,
+  "date": "2025-03-30T23:11:00",
+  "classification": "TrueNegative"
+},
+ ]

LLM/Database/README.md

@@ -0,0 +1,8 @@
+## Messages.json
+原始数据
+## Manual_filtering_samples.json
+手工筛的
+## negative_samples.json
+ai辅助筛的负样本
+## positive_samples.json
+ai辅助筛的正样本

LLM/Database/TrueNegative_samples.json

@@ -0,0 +1,197 @@
+ [ 
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG陈G老师1010: 【招商银行】您本月办理分期还款可享限时2.3折优惠！打开掌上生活APP，搜“分期还款”立享折扣优惠！资格实时测评，拒收请回复R",
+  "app_name": "com.tencent.mm",
+  "message_id": 330231000,
+  "user_id": 67619984,
+  "date": "2025-03-30T23:10:59",
+  "classification": "TrueNegative"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[4条]AlisaGG: 【招商银行】您本月办理分期还款可享限时2.3折优惠！打开掌上生活APP，搜“分期还款”立享折扣优惠！资格实时测评，拒收请回复R",
+  "app_name": "com.tencent.mm",
+  "message_id": 330231005,
+  "user_id": 74797059,
+  "date": "2025-03-30T23:11:00",
+  "classification": "TrueNegative"
+},
+{
+  "sender": "ASAP Azure ToDoAgent",
+  "content": "WANG Siyu: [图片]",
+  "app_name": "com.ss.android.lark",
+  "message_id": 401101707,
+  "user_id": 67619984,
+  "date": "2025-04-01T10:17:31"
+},
+{
+  "sender": "ASAP Azure ToDoAgent",
+  "content": "WANG Siyu: 我真的尝试给我们做一个公共邮箱 outlook🤮",
+  "app_name": "com.ss.android.lark",
+  "message_id": 401101708,
+  "user_id": 67619984,
+  "date": "2025-04-01T10:17:48"
+},
+{
+    "sender": "1065812627",
+    "content": "【流量领取提醒】尊敬的用户，您已获得10GB免费通用流量，将于2025年4月30日到期，点击：https://c.139.com/m/a/s?p=GDCM 或到“移动云盘App-我的-领奖专区-优惠卡券”领取，数量有限，领完即止。拒收请回复R【中国移动云盘】",
+    "app_name": "SMS",
+    "message_id": 401101418,
+    "user_id": 74797059,
+    "date": "2025-04-01T10:14:54"
+},
+{
+  "sender": "106575329994",
+  "content": "【霸王茶姬】好茶一口鲜，唤醒好状态，送你「整单8折+双杯78券」3ct.cc/e1mwQaSz 仅限广东区域部分门店核销 拒收请回复R",
+  "app_name": "SMS",
+  "message_id": 401100317,
+  "user_id": 74797059,
+  "date": "2025-04-01T10:03:40"
+},
+{
+  "sender": "10086",
+  "content": "尊敬的客户，根据您的充值历史记录赠送6.00元，您总共还有待赠送金额60.00元，其中下月将到帐6.00元，当前可用余额73.51元，感谢您对我公司的支持。登录“中国移动”APP足不出户查办业务，尊享充值优惠，立即前往： https://dx.10086.cn/A/kHXkEQ 。【中国移动】",
+  "app_name": "SMS",
+  "message_id": 401090015,
+  "user_id": 74797059,
+  "date": "2025-04-01T09:00:45"
+},
+{
+  "sender": "106980095533",
+  "content": "【建设银行】建行送福利啦！支付1分钱，有机会抽取2-188元微信立减金，100%中奖！手机银行APP首页搜索进入“建粤有礼”，点击“月月有礼”即可参与活动。如已参与，请忽略本条短信，详询广州建行各网点大堂经理。拒收请回复R",
+  "app_name": "SMS",
+  "message_id": 329111118,
+  "user_id": 67619984,
+  "date": "2025-03-29T11:11:51"
+},
+{
+  "sender": "1068518895595",
+  "content": "【光大银行】尊敬的光大信用卡受邀客户！欢迎参与4月积分福利大放送活动。2025年4月1日至30日累计消费满200元额外获赠1千积分，累计消费满800元再赠4千积分，累计消费满3千元叠加获赠1万积分！最高奖励1.5万积分，有效期1年。详情可咨询95595 。*拒收请回复R",
+  "app_name": "SMS",
+  "message_id": 328124936,
+  "user_id": 67619984,
+  "date": "2025-03-28T12:49:56"
+},
+{
+  "sender": "备忘录",
+  "content": "[4条]M丶D: 【开心麻花】开心麻花推出“降温补贴票”！《贼想得到你》新开场次买一送一，优惠限时到气温恢复15°C！抢票戳：kxmh.net/3Eo9。拒收请回复R",
+  "app_name": "com.tencent.mm",
+  "message_id": 328111725,
+  "user_id": 67619984,
+  "date": "2025-03-28T11:17:56"
+},
+{
+  "sender": "10086109",
+  "content": "春风送暖绿意浓，森林防火莫放松！清明祭祀倡新风，鲜花追思情更重；踏青旅游守规章，莫留火种在山中。严防森林火灾，永葆绿色家园。(市规划资源局)",
+  "app_name": "SMS",
+  "message_id": 328105821,
+  "user_id": 67619984,
+  "date": "2025-03-28T10:58:47"
+},
+{
+  "sender": "10680175780000628",
+  "content": "【养生课】派送中：通知您数次了，您的免费中医养生课程已经送到，请及时通过 b.i7o.cn/0B7YCY1 获取，拒收请回复R",
+  "app_name": "SMS",
+  "message_id": 328100301,
+  "user_id": 67619984,
+  "date": "2025-03-28T10:03:20"
+},
+{
+  "sender": "ASAP Azure ToDoAgent",
+  "content": "WANG Siyu: Message_id: 1, content:“ 【中国联通】出境漫游低至19.5元/天，订购天数越多越优惠，覆盖全球106个国家和地区，一次订购畅游全球！即日起至12月31日，订购国漫畅游流量3天流量包及以上天数产品，即可获赠权益六选一通兑券一张，打车、外卖、美食代金券等你来拿。订购方式点击https://u.10010.cn/uAbdw 或发送KTGM10至10010，根据回复提示进行订购，更多出境漫游流量产品请点击https://u.10010.cn...",
+  "app_name": "com.ss.android.lark",
+  "message_id": 327121407,
+  "user_id": 33642157,
+  "date": "2025-03-27T12:14:17"
+},
+{
+  "sender": "10086",
+  "content": "【防诈温馨提醒】尊敬的客户，惠州移动提醒您：“不明来电及时挂”、“可疑链接不点击”，请您对各类调研评分、评价有奖等陌生来电谨慎接听，谨防电信网络诈骗。您可回复KTFSR办理高频骚扰电话拦截服务，该服务无需收费，办理后如需取消请发送QXFSR到10086。点击 https://dx.10086.cn/A/tx04HA 进入“中国移动”APP服务大厅，可清晰查阅已办理业务，诚邀体验。心级服务、让爱连接，我们期待您的“10分”满意。【中国移动】",
+  "app_name": "SMS",
+  "message_id": 327111797,
+  "user_id": 74797059,
+  "date": "2025-03-27T11:17:29"
+},
+{
+  "sender": "🌳广州新地标！云山天地",
+  "content": "白云山南门旁，美食购物一站式，负一层还有儿童乐园，逛完白云山来歇歇脚吧！",
+  "app_name": "com.dianping.v1",
+  "message_id": 327085555,
+  "user_id": 67619984,
+  "date": "2025-03-27T08:55:56"
+},
+{
+  "sender": "四川一小区疑高空抛物砸死快递员",
+  "content": "点击查看详情>>>",
+  "app_name": "com.sina.weibo",
+  "message_id": 327083854,
+  "user_id": 67619984,
+  "date": "2025-03-27T08:38:49"
+},
+{
+  "sender": "男子为发明永动机出租屋连杀5人",
+  "content": "男子为发明永动机，想通过杀人锻炼胆量，在出租屋内连杀5人，被判处死刑，目前案件正在二审审理期间。",
+  "app_name": "com.ss.android.ugc.aweme",
+  "message_id": 327083851,
+  "user_id": 67619984,
+  "date": "2025-03-27T08:38:49"
+},
+{
+  "sender": "福利红包天天抢，最高666元",
+  "content": "机会难得，不要错过>>",
+  "app_name": "com.taobao.taobao",
+  "message_id": 327083853,
+  "user_id": 67619984,
+  "date": "2025-03-27T08:38:49"
+},
+{
+  "sender": "106980095533",
+  "content": "【建设银行】您账户8699于3月27日8时3分向微信支付-羊城通缴费支出人民币3.5元,可用余额5240.93元。",
+  "app_name": "SMS",
+  "message_id": 327080344,
+  "user_id": 67619984,
+  "date": "2025-03-27T08:03:48"
+},
+{
+  "sender": "恭喜，你被喜马选中做有声书副业！",
+  "content": "根据收听记录，你被认定适合喜马的声音副业，下班后用1-2个小时录制时薪书、分成书上传就可能获得收获！",
+  "app_name": "com.ximalaya.ting.android",
+  "message_id": 326202922,
+  "user_id": 67619984,
+  "date": "2025-03-26T20:29:01"
+},
+{
+  "sender": "备忘录",
+  "content": "[14条]M丶D: [文件] WEMEC METABOTS R60(1).pdf",
+  "app_name": "com.tencent.mm",
+  "message_id": 326183714,
+  "user_id": 67619984,
+  "date": "2025-03-26T18:37:48"
+},
+{
+  "sender": "你的兴趣商品",
+  "content": "JETSON ORIN NX Super 开发套件 AI 智能模组orin nx 主板限时优惠发放中！",
+  "app_name": "com.taobao.taobao",
+  "message_id": 326190015,
+  "user_id": 67619984,
+  "date": "2025-03-26T19:00:09"
+},
+{
+  "sender": "恭喜您被666元红包选中啦！",
+  "content": "好友***已成功领取，抓紧参与吧!",
+  "app_name": "com.taobao.taobao",
+  "message_id": 326190116,
+  "user_id": 67619984,
+  "date": "2025-03-26T19:01:14"
+},
+{
+  "sender": "106980095533",
+  "content": "【建设银行】您账户8699于3月26日19时8分向微信支付-羊城通缴费支出人民币1元,可用余额5263.83元。",
+  "app_name": "SMS",
+  "message_id": 326190917,
+  "user_id": 67619984,
+  "date": "2025-03-26T19:09:07"
+},
+
+ ]

LLM/Database/TruePositive_samples.json

@@ -0,0 +1,270 @@
+ [ 
+{
+    "sender": "ASAP Sample",
+    "content": "王斯煜[表情]Vince 黑客松nv: @AlisaGG @LJK86 明天中午再对 WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331001504,
+    "user_id": 67619984,
+    "date": "2025-03-31T00:15:21",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]王斯煜[表情]Vince 黑客松nv: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002905,
+    "user_id": 67619984,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[4条]斯煜[表情]Vince: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002907,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[4条]斯煜[表情]Vince: 【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002908,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]斯煜[表情]Vince: 尊敬的*斯煜，您在我行办理的1笔个人贷款需于2024年12月08日17:00前还款，当期还款金额本息合计999999.84元，请您留意尾号0455的账户可用余额是否充足，避免因贷款逾期影响个人征信。具体贷款信息可通过工行网上银行、手机银行或致电贷款经办行查询。【工商银行】",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002909,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+    "sender": "ASAP Sample",
+    "content": "[6条]斯煜[表情]Vince: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se...",
+    "app_name": "com.tencent.mm",
+    "message_id": 331002910,
+    "user_id": 74797059,
+    "date": "2025-03-31T00:29:45",
+    "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "王斯煜[表情]Vince 黑客松nv: WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://vc.feishu.cn/j/65644590...",
+  "app_name": "com.tencent.mm",
+  "message_id": 331115637,
+  "user_id": 67619984,
+  "date": "2025-03-31T11:56:36",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG陈G老师1010: 【驿收发】您的邮政包裹已到凯丰花园2栋驿站，请23:00前凭5-5-6530来取，详询18320926368",
+  "app_name": "com.tencent.mm",
+  "message_id": 331133944,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:39:49",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[2条]AlisaGG: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134011,
+  "user_id": 74797059,
+  "date": "2025-03-31T13:40:54"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[2条]AlisaGG陈G老师1010: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134045,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:40:54",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[3条]AlisaGG陈G老师1010: 取件通知\n取件码: 11724837\n运单号: 464285154986072\n取件地址: 深圳新一代产业园P2出入口内侧4号丰巢柜\n配送人员: 18124519013\n计费规则: 查看详情",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134346,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:43:17",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[4条]AlisaGG陈G老师1010: 取件再次提醒\n取件码：: 27696201\n配送公司：: 申通快递\n运单号：: 777293635831671\n配送���手机：: 13392809673\n取件地址：: 深圳新一代产业园P2出入口内侧4号丰巢柜",
+  "app_name": "com.tencent.mm",
+  "message_id": 331134447,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:44:02",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[5条]李JK老师-1228: 【菜鸟驿站】请凭140-3-1019到菜鸟驿站取件，查询详情u.cainiao.com/53h4bSr7zrh",
+  "app_name": "com.tencent.mm",
+  "message_id": 331135751,
+  "user_id": 67619984,
+  "date": "2025-03-31T13:57:28",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "刻刻: 【停机前提醒】尊敬的移动客户，您好！您的账户余额不足被限制使用。现提醒您需充值缴费至少32.49元，以确保您继续享受畅通的通信服务。诚邀您一键办理自动充服务，自动充值缴费更轻松：https://dx.10086.cn/7WyRLA。 心级服务、让爱连接【中国移动】",
+  "app_name": "com.tencent.mm",
+  "message_id": 331162269,
+  "user_id": 67619984,
+  "date": "2025-03-31T16:22:21",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[3条]LJK86: 您的号卡已在配送途中，物流单号SF3147624215612,点击：http://t.hn.189.cn/EZv6bazD，查询物流进度。如已签收请在有效期内先实名激活。客服热线4008155555【中国电信】",
+  "app_name": "com.tencent.mm",
+  "message_id": 330225703,
+  "user_id": 74797059,
+  "date": "2025-03-30T22:57:51",
+  "classification": "TruePositive"
+},
+{
+  "sender": "106875230196298038",
+  "content": "【京东快递】取件码C8042，您的快件尾号8980已送达成都东软学院配送营业点，地址：四川成都市都江堰市青城山镇东软大道1号东软学院电信营业厅旁 ，联系电话13032837084",
+  "app_name": "SMS",
+  "message_id": 328233028,
+  "user_id": 33642157,
+  "date": "2025-03-28T23:30:29",
+  "classification": "TruePositive"
+},
+{
+  "sender": "1068837016151",
+  "content": "【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。",
+  "app_name": "SMS",
+  "message_id": 322223721,
+  "user_id": 67619984,
+  "date": "2025-03-22T22:37:50",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Azure ToDoAgent",
+  "content": "陈格(alisagege.chen): WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://vc.feishu.cn/j/656445907\n\n手机拨号一键入会\n+862122504720,,656445907（中国大陆）\n4008208888,,656445907（中国大陆）\n\n根据所在地拨打号...",
+  "app_name": "com.ss.android.lark",
+  "message_id": 331150106,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:36",
+  "classification": "TruePositive"
+},
+{
+  "sender": "106917190196",
+  "content": "【广州自来水】(水费通知)尊敬的0001914043用户（大沙头三马路9号大院8号604）水量6（202503行度83），应缴水费11.88元，污水费5.7元。了解更多用水资讯可关注“广州自来水96968”微信公众号。",
+  "app_name": "SMS",
+  "message_id": 319155403,
+  "user_id": 67619984,
+  "date": "2025-03-19T15:54:16",
+  "classification": "TruePositive"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG: 【小象超市】您的商品已放置在门口，因有易碎等商品请尽快取回，如有疑问请联系15794935204 。祝您生活愉快！",
+  "app_name": "com.tencent.mm",
+  "message_id": 331232914,
+  "user_id": 74797059,
+  "date": "2025-03-31T23:29:47"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG陈G老师1010: 【小象超市】您的商品已放置在门口，因有易碎等商品请尽快取回，如有疑问请联系15794935204 。祝您生活愉快！",
+  "app_name": "com.tencent.mm",
+  "message_id": 331232986,
+  "user_id": 67619984,
+  "date": "2025-03-31T23:29:46"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[3条]AlisaGG: 取件通知\n取件码: 11724837\n运单号: 464285154986072\n取件地址: 深圳新一代产业园P2出入口内侧4号丰巢柜\n配送人员: 18124519013\n计费规则: 查看详情",
+  "app_name": "com.tencent.mm",
+  "message_id": 331150113,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:37"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[4条]AlisaGG: 取件再次提醒\n取件码：: 27696201\n配送公司：: 申通快递\n运单号：: 777293635831671\n配送员手机：: 13392809673\n取件地址：: 深圳新一代产业园P2出入口内侧4号丰巢柜",
+  "app_name": "com.tencent.mm",
+  "message_id": 331150114,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:37"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "AlisaGG: 【驿收发】您的邮政包裹已到凯丰花园2栋驿站，请23:00前凭5-5-6530来取，详询18320926368",
+  "app_name": "com.tencent.mm",
+  "message_id": 331150111,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:36"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "[2条]AlisaGG: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。",
+  "app_name": "com.tencent.mm",
+  "message_id": 331150112,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:37"
+},
+{
+  "sender": "ASAP Sample",
+  "content": "斯煜[表情]Vince: WANG Siyu邀请你加入飞书视频会议\n会议主题：FilterLLM ToDoGenLLM PR Merge讨论\n会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)\n会议 ID：656 445 907\n会议链接：https://vc.feishu.cn/j/65644590...",
+  "app_name": "com.tencent.mm",
+  "message_id": 331150102,
+  "user_id": 33642157,
+  "date": "2025-03-31T15:01:37"
+},
+{
+  "sender": "#SoSIM",
+  "content": "[SoSIM提示] 未來30日自動續購服務收費 (04/2025)\r\n\r\n為確保你現有的服務不會中斷，根據系統紀錄，以下是你未來30日已開啟自動續購的服務收費總額，只供參考。如果此額多於「現有儲值額」，建議立即增值。\r\n\r\n\r\n儲值卡號碼：96334767\r\n\r\n未來30日續購服務總額 : $55.0\r\n\r\n現有儲值額 : $26.65\r\n\r\n注意：個別服務 (包括內地副號及附屬卡) 一旦到期而未有足夠餘額自動續購，有關號碼將即時失效而不可復原。\r\n\r\n請於賬戶主頁 > 設定 內瀏覽有關服務組合有效期及詳情。\r\n\r\n\r\n***如果你的「自動增值」金額不少於所需服務收費，請忽略此訊息***",
+  "app_name": "SMS",
+  "message_id": 401100763,
+  "user_id": 96989258,
+  "date": "2025-04-01T10:07:29"
+},
+{
+  "sender": "106875230196298038",
+  "content": "【京东快递】取件码C8042，您的快件尾号8980已送达成都东软学院配送营业点，地址：四川成都市都江堰市青城山镇东软大道1号东软学院电信营业厅旁 ，联系电话13032837084",
+  "app_name": "SMS",
+  "message_id": 328233028,
+  "user_id": 33642157,
+  "date": "2025-03-28T23:30:29"
+},
+{
+  "sender": "您购买的宝贝已送达自提柜",
+  "content": "包裹待签收，点击查看详情>>",
+  "app_name": "com.taobao.taobao",
+  "message_id": 327135963,
+  "user_id": 67619984,
+  "date": "2025-03-27T13:59:30"
+},
+{
+  "sender": "您购买的宝贝正在派送中",
+  "content": "【搓澡巾洗澡神器男女款】预计送往【智能柜】，手机号已加密，享号码保护服务，点击查看预计送达时间>>",
+  "app_name": "com.taobao.taobao",
+  "message_id": 327091156,
+  "user_id": 67619984,
+  "date": "2025-03-27T09:11:37"
+},
+{
+  "sender": "#SoSIM",
+  "content": "[SoSIM提示] 30日/無限影視數據組合 (由2025年1月7日起續購收費$55/30日) 用量提示：\r\n儲值卡號碼：96334767\r\n累積用量：30 GB \r\n公平使用數據用量上限：已用50% (截至：27/03/2025 06:53:27)\r\n如已達到 / 超過公平使用數據用量100%，數據傳輸速度將會即時被限制至高達128kbps，直至 25/04/2025 19:42:52 (香港時間)。\r\n立即於賬戶主頁 > 本地服務 https://sosimhk.com/SoSim/main/tc/topup 購買追加數據以保持流暢的上網速度。\r\n",
+  "app_name": "SMS",
+  "message_id": 327065312,
+  "user_id": 96989258,
+  "date": "2025-03-27T06:53:42"
+},
+
+
+ ]

LLM/Database/classify_samples.py

@@ -0,0 +1,222 @@
+import json
+import csv
+import os
+import pandas as pd
+import openai
+import time
+import requests
+from dotenv import load_dotenv
+from tqdm import tqdm
+
+# 加载环境变量（如果有.env文件）
+load_dotenv()
+
+# 配置SiliconFlow API
+SILICONFLOW_API_KEY = os.getenv("SILICONFLOW_API_KEY", "sk-ypjvmantsostdxrkirhidrtswohjpmlzuhyqojpudbreakwk")
+SILICONFLOW_API_BASE = os.getenv("SILICONFLOW_API_BASE", "https://api.siliconflow.cn/v1")
+
+# 保留OpenAI API配置（作为备选）
+OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "your_openai_api_key_here")
+openai.api_key = OPENAI_API_KEY
+
+# 可以配置为Azure OpenAI
+#AZURE_OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT", "")
+#AZURE_OPENAI_API_KEY = os.getenv("AZURE_OPENAI_API_KEY", "")
+#AZURE_DEPLOYMENT_NAME = os.getenv("AZURE_DEPLOYMENT_NAME", "")
+
+# 获取Azure配置参数
+AZURE_OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT", "")
+AZURE_OPENAI_API_KEY = os.getenv("AZURE_OPENAI_API_KEY", "")
+AZURE_DEPLOYMENT_NAME = os.getenv("AZURE_DEPLOYMENT_NAME", "")
+
+# 如果有Azure OpenAI配置，则使用Azure OpenAI
+if AZURE_OPENAI_ENDPOINT.strip() and AZURE_OPENAI_API_KEY.strip() and AZURE_DEPLOYMENT_NAME.strip():
+    openai.api_type = "azure"
+    openai.api_base = AZURE_OPENAI_ENDPOINT
+    openai.api_key = AZURE_OPENAI_API_KEY
+    openai.api_version = "2023-05-15"  # 可能需要根据实际情况调整
+
+# 定义TruePositive的标准（根据mvp三类案例）
+def define_positive_sample_criteria():
+    """
+    定义TruePositive的标准
+    根据搜索结果，TruePositive被定义为"mvp三类案例"，但没有找到具体定义
+    这里我们定义一些可能的标准，实际使用时可以根据需求调整
+    """
+    return """
+    请判断以下消息是否属于TruePositive。TruePositive定义为与任务管理、待办事项、提醒、通知筛选相关的有用信息，具体包括：
+    1. 包含明确的任务、待办事项或需要完成的工作
+    2. 包含时间安排、截止日期或日程提醒
+    3. 包含项目进展、状态更新或工作报告
+    
+    如果消息符合以上任一条件，则为TruePositive；否则为TrueNegative。
+    请只回答"TruePositive"或"TrueNegative"。
+    """
+
+# 使用大模型API进行分类
+def classify_with_llm(message, criteria, max_retries=3, retry_delay=2):
+    """
+    使用大模型API对消息进行分类
+    
+    Args:
+        message: 要分类的消息内容
+        criteria: 分类标准
+        max_retries: 最大重试次数
+        retry_delay: 重试延迟（秒）
+        
+    Returns:
+        str: "TruePositive" 或 "TrueNegative"
+    """
+    prompt = f"{criteria}\n\n消息内容: {message}"
+    system_message = "你是一个专业的数据分类助手，根据给定标准判断消息是TruePositive还是TrueNegative。"
+    
+    for attempt in range(max_retries):
+        try:
+            # 使用SiliconFlow API
+            headers = {
+                "Content-Type": "application/json",
+                "Authorization": f"Bearer {SILICONFLOW_API_KEY}"
+            }
+            
+            payload = {
+                "model": "deepseek-ai/DeepSeek-V3",
+                "messages": [
+                    {"role": "system", "content": system_message},
+                    {"role": "user", "content": prompt}
+                ],
+                "stream": False,
+                "max_tokens": 512,
+                "temperature": 0.1,
+                "top_p": 0.7,
+                "top_k": 50,
+                "frequency_penalty": 0.5,
+                "n": 1
+            }
+            
+            response = requests.post(
+                f"{SILICONFLOW_API_BASE}/chat/completions",
+                headers=headers,
+                json=payload
+            )
+            
+            # 检查响应状态
+            response.raise_for_status()
+            response_data = response.json()
+            
+            # 解析响应
+            result = response_data["choices"][0]["message"]["content"].strip()
+            
+            # 标准化结果
+            if "TruePositive" in result:
+                return "TruePositive"
+            else:
+                return "TrueNegative"
+                
+        except Exception as e:
+            if attempt < max_retries - 1:
+                print(f"API调用失败，{retry_delay}秒后重试: {e}\n响应状态码: {response.status_code if 'response' in locals() else 'N/A'}\n响应内容: {response.text if 'response' in locals() else 'N/A'}")
+                time.sleep(retry_delay)
+            else:
+                print(f"API调用失败，达到最大重试次数: {e}\n最后响应状态码: {response.status_code if 'response' in locals() else 'N/A'}\n最后响应内容: {response.text if 'response' in locals() else 'N/A'}")
+                return "分类失败"  # 返回一个默认值
+
+# 批量处理消息
+def batch_process_messages(messages, batch_size=10, delay=1):
+    """
+    批量处理消息以避免API限制
+    
+    Args:
+        messages: 消息列表
+        batch_size: 每批处理的消息数量
+        delay: 批次间延迟（秒）
+        
+    Returns:
+        list: 处理结果列表
+    """
+    results = []
+    criteria = define_positive_sample_criteria()
+    
+    for i in tqdm(range(0, len(messages), batch_size), desc="处理批次"):
+        batch = messages[i:i+batch_size]
+        batch_results = []
+        
+        for msg in tqdm(batch, desc="处理消息", leave=False):
+            # 只处理有实际内容的消息
+            if msg.get("content") and len(msg["content"]) > 5:  # 忽略过短的消息
+                classification = classify_with_llm(msg["content"], criteria)
+                msg["classification"] = classification
+            else:
+                msg["classification"] = "TrueNegative"  # 默认短消息为TrueNegative
+                
+            batch_results.append(msg)
+            
+        results.extend(batch_results)
+        
+        if i + batch_size < len(messages):
+            time.sleep(delay)  # 批次间延迟
+            
+    return results
+
+# 主函数
+def main():
+    # 检查API密钥是否配置
+    if SILICONFLOW_API_KEY == "":
+        print("警告: 未设置SiliconFlow API密钥。请设置环境变量SILICONFLOW_API_KEY或在代码中直接设置。")
+        return
+    
+    # 确定输入文件
+    input_file = "Messages.json"  # 默认使用JSON格式
+    if not os.path.exists(input_file):
+        print(f"错误: 找不到JSON输入文件 {input_file}")
+        return
+    
+    print(f"使用输入文件: {input_file}")
+    
+    # 读取数据
+    messages = []
+    if input_file.endswith(".json"):
+        with open(input_file, "r", encoding="utf-8") as f:
+            messages = json.load(f)
+
+    
+    print(f"读取了 {len(messages)} 条消息")
+    
+    # 询问用户是否要处理所有消息或仅处理一部分样本
+    sample_size = input("请输入要处理的消息数量（输入'all'处理所有消息，或输入一个数字如'100'处理部分消息）: ")
+    
+    if sample_size.lower() != "all":
+        try:
+            sample_size = int(sample_size)
+            if sample_size < len(messages):
+                print(f"将处理 {sample_size} 条消息作为样本")
+                messages = messages[:sample_size]
+            else:
+                print(f"样本大小大于等于总消息数，将处理所有 {len(messages)} 条消息")
+        except ValueError:
+            print("无效输入，将处理所有消息")
+    
+    # 批量处理消息
+    print("开始处理消息...")
+    classified_messages = batch_process_messages(messages)
+    
+    # 分离TruePositive from TrueNegative
+    positive_samples = [msg for msg in classified_messages if msg.get("classification") == "TruePositive"]
+    negative_samples = [msg for msg in classified_messages if msg.get("classification") == "TrueNegative"]
+    
+    print(f"分类完成: TruePositive {len(positive_samples)} 条, TrueNegative {len(negative_samples)} 条")
+    
+    # 保存结果
+    if input_file.endswith(".json"):
+        # 保存JSON格式
+        with open("positive_samples.json", "w", encoding="utf-8") as f:
+            json.dump(positive_samples, f, ensure_ascii=False, indent=2)
+        
+        with open("negative_samples.json", "w", encoding="utf-8") as f:
+            json.dump(negative_samples, f, ensure_ascii=False, indent=2)
+    
+
+    
+    print("结果已保存到 positive_samples.json/csv 和 negative_samples.json/csv")
+
+if __name__ == "__main__":
+    main()

LLM/Database/negative_samples.json

@@ -0,0 +1 @@
+[]

LLM/Database/update_Messagejson.py

@@ -0,0 +1,63 @@
+import mysql.connector
+from datetime import datetime  # 导入 datetime 模块
+import os
+from pathlib import Path
+import json
+
+#Azure MySQL数据库连接
+current_dir = Path(__file__).parent.absolute()
+ssl_ca_path = current_dir / "DigiCertGlobalRootCA.crt.pem"
+
+#写入json时对datetime类型进行序列化
+def datetime_serializer(obj):
+    if isinstance(obj, datetime):
+        return obj.isoformat()  # 将 datetime 转换为 ISO 8601 格式的字符串
+    raise TypeError("Type not serializable")
+
+try:
+    # 建立数据库连接
+    cnx = mysql.connector.connect(
+        user="siyuwang541", 
+        password="ToDoAgentASAP！1", 
+        host="todoagent-databases.mysql.database.azure.com", 
+        port=3306, 
+        database="todoagent",
+        ssl_ca=str(ssl_ca_path),
+        ssl_disabled=False
+    )
+
+    print("数据库连接成功！")
+    
+    # 测试查询
+    cursor = cnx.cursor()
+    cursor.execute("SELECT * FROM Messages")
+    # 获取表头（列名）
+    columns = [desc[0] for desc in cursor.description]
+
+    # 获取数据
+    rows = cursor.fetchall()
+
+    # 将表头和数据合并为字典列表
+    data = [dict(zip(columns, row)) for row in rows]
+
+    # 打印表头和数据
+    print("表头:", columns)
+    print("数据:")
+    for row in data:
+        print(row)
+
+    # 将数据写入 JSON 文件
+    with open("Messages.json", "w", encoding="utf-8") as file:
+        json.dump(data, file, ensure_ascii=False, indent=4, default=datetime_serializer)  # datetime使用自定义序列化器
+
+    # 关闭连接
+    cursor.close()
+    cnx.close()
+    print("连接已正常关闭")
+    
+except mysql.connector.Error as err:
+    print(f"数据库错误: {err}")
+except Exception as e:
+    print(f"发生异常: {str(e)}")
+
+    

LLM/Notify/NotifyReadme.md

@@ -0,0 +1,50 @@
+<!--
+ * @Description: 
+ * @Author: Manda
+ * @Version: 
+ * @Date: 2025-03-30 17:01:58
+ * @LastEditors: Manda
+ * @LastEditTime: 2025-04-03 13:39:35
+-->
+# 运行方式
+确保DigiCertGlobalRootCA.crt.pem与config.yaml在同一文件夹内
+运行方式 pip install -r requirements.txt   按装环境
+运行 python notifyMain.py
+
+## 20250403
+### 更新数据库连接办法
+启用Azure 与 GG老师的 yaml
+更新requirements.txt，增加yaml
+
+## 20250402
+### notifyMain.py 主函数
+主函数，每隔1小时调用下方三个py('db2txt.py', 'usrSpareTime.py', 'compareDb2txt.py')
+运行方式 pip install -r requirements.txt   按装环境
+运行 python notifyMain.py
+-----------------------------
+得到的效果是：
+生成compare_output，output，time_analysis 文件夹
+#### compare_output
+对比相同user_ID的UCtodolist表 & todolist表 生成结果
+#### output
+拉取 todolist表 生成user_ID.txt
+#### time_analysis
+对比相同user_ID的UCtodolist表不同更改时段，生成结果
+
+## 20250320 
+一些做RAG的数据文档以user_id来命名
+
+### dataBaseConnecter
+dataBaseConnecter.py实现连接服务器功能，并提供端口让其他py（如db2txt.py）将指定数据库内文本提取出来
+
+### db2txt （好像直接在数据库完成了对比，这个文件貌似没啥用了）
+db2txt.py 将ToDoAgent数据库中的ToDoList表格内容下载到txt中，按ToDoList表格中的user_id来命名txt,也就是不同的user_id有不同的txt
+
+### usrSpareTime -->千人千面推送时间可以用到的RAG
+usrSpareTime.py 将ToDoAgent数据库中的UCtodolist表格内容last_modified，数据获取出来分36个时段进行统计，统计出出现频率最高的6个时段， 将时段信息及出现次数下载到txt中，以相同“todo_id”为前提，查询ToDoList表格中的user_id来命名txt,也就是不同的user_id有不同的txt
+
+### compareDb2txt-->自动生成ToDoList可以用到的的RAG
+compareDb2txt.py 将ToDoAgent数据库中的UCtodolist表格内容与ToDoList做对比，以相同“todo_id”为前提，对比“start_time”"end_time""location""todo_content",一旦发现有差异,则将差异内容下载到txt中，按ToDoList表格中的user_id来命名txt,也就是不同的user_id有不同的txt
+
+
+

LLM/Notify/compareDb2txt.py

@@ -0,0 +1,130 @@
+# '''
+# Description: 
+# Author: Manda
+# Version: 
+# Date: 2025-03-30 16:28:58
+# LastEditors: mdhuang555 [email protected]
+# LastEditTime: 2025-03-30 16:39:18
+# '''
+from dataBaseConnecter import DatabaseConnector
+import os
+from datetime import datetime
+import sys
+import io
+sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
+
+def get_table_data(db_connector: DatabaseConnector, table_name: str) -> dict:
+    """获取表格数据，以todo_id为键"""
+    try:
+        # 连接数据库
+        conn = db_connector.connect_db()
+        if not conn:
+            print("无法连接到数据库")
+            return {}
+            
+        cursor = conn.cursor(dictionary=True)
+        try:
+            # 使用连接器的extract_text方法获取数据
+            results = db_connector.extract_text(conn, table_name, '*')
+            # 将结果转换为以todo_id为键的字典
+            return {str(row['todo_id']): row for row in results}
+        finally:
+            cursor.close()
+            conn.close()
+    except Exception as e:
+        print(f"获取{table_name}数据错误: {e}")
+        return {}
+
+def compare_records(todolist_record: dict, uctodolist_record: dict) -> dict:
+    """比较两条记录的差异"""
+    differences = {}
+    fields_to_compare = ['start_time', 'end_time', 'location', 'todo_content']
+    
+    for field in fields_to_compare:
+        todo_value = todolist_record.get(field)
+        uc_value = uctodolist_record.get(field)
+        
+        # 特殊处理datetime类型的比较
+        if isinstance(todo_value, datetime):
+            todo_value = todo_value.strftime('%Y-%m-%d %H:%M:%S')
+        if isinstance(uc_value, datetime):
+            uc_value = uc_value.strftime('%Y-%m-%d %H:%M:%S')
+            
+        if todo_value != uc_value:
+            differences[field] = {
+                'ToDoList': todo_value,
+                'UCtodolist': uc_value
+            }
+    
+    return differences
+
+def save_differences_to_file(differences: dict, output_dir: str = 'compare_output'):
+    """将差异保存到文件中"""
+    if not os.path.exists(output_dir):
+        os.makedirs(output_dir)
+    
+    # 按用户ID分组
+    user_differences = {}
+    for todo_id, diff in differences.items():
+        user_id = diff['user_id']
+        if user_id not in user_differences:
+            user_differences[user_id] = {}
+        user_differences[user_id][todo_id] = diff['differences']
+
+    # 为每个用户创建文件
+    for user_id, user_diffs in user_differences.items():
+        filename = os.path.join(output_dir, f'user_{user_id}_differences.txt')
+        with open(filename, 'w', encoding='utf-8') as f:
+            f.write(f"对比时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
+            f.write(f"用户ID: {user_id}\n")
+            f.write("=" * 50 + "\n\n")
+            
+            for todo_id, diffs in user_diffs.items():
+                f.write(f"待办事项ID: {todo_id}\n")
+                for field, values in diffs.items():
+                    f.write(f"  字段: {field}\n")
+                    f.write(f"    ToDoList值: {values['ToDoList']}\n")
+                    f.write(f"    UCtodolist值: {values['UCtodolist']}\n")
+                f.write("-" * 50 + "\n")
+        
+        print(f"已保存用户 {user_id} 的差异到文件: {filename}")
+
+def main():
+    print("正在连接数据库...")
+    
+    try:
+        # 创建数据库连接器实例
+        db_connector = DatabaseConnector()
+        
+        # 获取两个表的数据
+        print("正在获取表格数据...")
+        todolist_data = get_table_data(db_connector, 'ToDoList')
+        uctodolist_data = get_table_data(db_connector, 'UCtodolist')
+
+        # 比较差异
+        print("正在比较差异...")
+        differences = {}
+        for todo_id in set(todolist_data.keys()) & set(uctodolist_data.keys()):
+            todolist_record = todolist_data[todo_id]
+            uctodolist_record = uctodolist_data[todo_id]
+            
+            record_differences = compare_records(todolist_record, uctodolist_record)
+            if record_differences:
+                differences[todo_id] = {
+                    'user_id': todolist_record['user_id'],
+                    'differences': record_differences
+                }
+
+        # 保存差异
+        if differences:
+            print(f"发现 {len(differences)} 条记录有差异")
+            save_differences_to_file(differences)
+            print("差异已保存到文件中")
+        else:
+            print("未发现差异")
+
+    except Exception as e:
+        print(f"处理过程中出错: {e}")
+
+if __name__ == "__main__":
+    main()

LLM/Notify/dataBaseConnecter.py

@@ -0,0 +1,128 @@
+# '''
+# Author: mdhuang555 [email protected]
+# Date: 2025-03-30 15:57:22
+# LastEditors: mdhuang555 [email protected]
+# LastEditTime: 2025-04-03 11:32:30
+# FilePath: \Notyif\dataBaseConnecter.py
+# Description: 数据库连接器，支持SSL连接
+# '''
+import socket
+import json
+import mysql.connector
+from typing import Dict, Any, Optional
+import yaml
+from pathlib import Path
+
+class DatabaseConnector:
+    def __init__(self, host: str = '103.116.245.150', port: int = 3306):
+        self.host = host
+        self.port = port
+        self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.config = self._load_config()
+        
+    def _load_config(self) -> Dict[str, Any]:
+        """加载配置文件"""
+        try:
+            config_path = Path(__file__).parent / "config.yaml"
+            with open(config_path, "r", encoding="utf-8") as f:
+                return yaml.safe_load(f)
+        except Exception as e:
+            print(f"加载配置文件错误: {e}")
+            return {}
+        
+    def connect_db(self) -> Optional[mysql.connector.MySQLConnection]:
+        """连接到MySQL数据库，使用SSL连接"""
+        try:
+            # 获取SSL证书路径
+            current_dir = Path(__file__).parent.absolute()
+            ssl_ca_path = current_dir / "DigiCertGlobalRootCA.crt.pem"
+            
+            # 确保SSL证书文件存在
+            if not ssl_ca_path.exists():
+                raise FileNotFoundError(f"SSL证书文件未找到: {ssl_ca_path}")
+            
+            # 建立数据库连接
+            conn = mysql.connector.connect(
+                host=self.config["mysql"]["host"],
+                port=self.config["mysql"].get("port", 3306),
+                user=self.config["mysql"]["user"],
+                password=self.config["mysql"]["password"],
+                database=self.config["mysql"]["database"],
+                ssl_ca=str(ssl_ca_path),
+                ssl_disabled=False,
+                charset='utf8mb4',
+                collation='utf8mb4_unicode_ci'
+            )
+            return conn
+        except Exception as e:
+            print(f"数据库连接错误: {e}")
+            return None
+
+    def extract_text(self, conn: mysql.connector.MySQLConnection, table: str, column: str) -> list:
+        """从指定表格和列中提取文本"""
+        try:
+            cursor = conn.cursor(dictionary=True)
+            # 如果请求所有列，则获取完整的行数据
+            if column == '*':
+                query = f"SELECT * FROM {table}"
+            else:
+                query = f"SELECT {column} FROM {table}"
+            cursor.execute(query)
+            results = cursor.fetchall()
+            cursor.close()
+            return results
+        except Exception as e:
+            print(f"提取文本错误: {e}")
+            return []
+
+    def start_server(self):
+        """启动服务器监听请求"""
+        self.server_socket.bind((self.host, self.port))
+        self.server_socket.listen(5)
+        print(f"服务器启动在 {self.host}:{self.port}")
+
+        while True:
+            try:
+                client_socket, address = self.server_socket.accept()
+                print(f"接受来自 {address} 的连接")
+                
+                # 接收客户端请求
+                data = client_socket.recv(1024).decode('utf-8')
+                request = json.loads(data)
+                
+                # 处理请求
+                table = request.get('table')
+                column = request.get('column')
+                
+                # 连接数据库并提取文本
+                conn = self.connect_db()
+                if conn:
+                    try:
+                        results = self.extract_text(conn, table, column)
+                        response = {'status': 'success', 'data': results}
+                    except Exception as e:
+                        response = {'status': 'error', 'message': str(e)}
+                    finally:
+                        conn.close()
+                else:
+                    response = {'status': 'error', 'message': '数据库连接失败'}
+                
+                # 发送响应
+                response_data = json.dumps(response, ensure_ascii=False)
+                response_bytes = response_data.encode('utf-8')
+                
+                # 先发送数据长度
+                length_prefix = len(response_bytes).to_bytes(4, byteorder='big')
+                client_socket.send(length_prefix)
+                
+                # 再发送实际数据
+                client_socket.send(response_bytes)
+                client_socket.close()
+                
+            except Exception as e:
+                print(f"处理请求错误: {e}")
+                continue
+
+if __name__ == "__main__":
+    server = DatabaseConnector()
+    server.start_server()

LLM/Notify/db2txt.py

@@ -0,0 +1,88 @@
+
+# '''
+# Author: mdhuang555 [email protected]
+# Date: 2025-03-30 16:09:29
+# LastEditors: mdhuang555 [email protected]
+# LastEditTime: 2025-04-03 11:02:35
+# FilePath: \Notify\db2txt.py
+# Description: 这是默认设置,请设置`customMade`, 打开koroFileHeader查看配置 进行设置: https://github.com/OBKoro1/koro1FileHeader/wiki/%E9%85%8D%E7%BD%AE
+# '''
+from dataBaseConnecter import DatabaseConnector
+import sys
+import io
+sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
+import os
+from datetime import datetime
+
+def get_database_text(table: str) -> list:
+    """使用DatabaseConnector从数据库获取数据"""
+    try:
+        # 创建数据库连接器实例
+        db_connector = DatabaseConnector()
+        
+        # 连接数据库
+        conn = db_connector.connect_db()
+        if not conn:
+            print("无法连接到数据库")
+            return []
+            
+        try:
+            # 使用连接器的extract_text方法获取数据
+            results = db_connector.extract_text(conn, table, '*')
+            return results
+        finally:
+            conn.close()
+            
+    except Exception as e:
+        print(f"获取数据时发生错误: {e}")
+        return []
+
+def save_todos_by_user(todos: list, output_dir: str = 'output'):
+    """将待办事项按用户ID保存到不同的文本文件中"""
+    if not os.path.exists(output_dir):
+        os.makedirs(output_dir)
+    
+    if not todos:
+        print("没有数据可以保存")
+        return
+    
+    # 按用户ID分组
+    user_todos = {}
+    for todo in todos:
+        user_id = str(todo['user_id'])
+        if user_id not in user_todos:
+            user_todos[user_id] = []
+        user_todos[user_id].append(todo)
+    
+    # 为每个用户创建文件
+    for user_id, user_todos_list in user_todos.items():
+        filename = os.path.join(output_dir, f'{user_id}.txt')
+        try:
+            with open(filename, 'w', encoding='utf-8') as f:
+                f.write(f"导出时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
+                f.write(f"用户ID: {user_id}\n")
+                f.write("=" * 50 + "\n\n")
+                
+                for todo in user_todos_list:
+                    f.write("待办事项:\n")
+                    for key, value in todo.items():
+                        if value is not None:  # 只写入非空值
+                            f.write(f"  {key}: {value}\n")
+                    f.write("-" * 50 + "\n")
+            print(f"已保存用户 {user_id} 的待办事项到文件: {filename}")
+        except Exception as e:
+            print(f"保存用户 {user_id} 的数据时出错: {e}")
+
+def main():
+    print("正在连接数据库...")
+    todos = get_database_text('ToDoList')
+    
+    if todos:
+        print(f"成功获取 {len(todos)} 条记录")
+        save_todos_by_user(todos)
+        print("所有数据已保存完成")
+    else:
+        print("未能获取到数据")
+
+if __name__ == "__main__":
+    main()

LLM/Notify/notifyMain.py

@@ -0,0 +1,88 @@
+
+# '''
+# Description: 定时运行数据库相关脚本的主程序
+# Author: Manda
+# Version: 1.0
+# Date: 2024-03-30
+# '''
+import schedule
+import time
+import subprocess
+import logging
+from datetime import datetime
+import os
+import sys
+import io
+sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
+
+# 配置日志
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(levelname)s - %(message)s',
+    handlers=[
+        logging.FileHandler('notify_main.log', encoding='utf-8'),
+        logging.StreamHandler()
+    ]
+)
+
+
+def run_script(script_name: str):
+    """运行指定的Python脚本"""
+    try:
+        logging.info(f"开始运行脚本: {script_name}")
+        result = subprocess.run(['python', script_name], 
+                              capture_output=True, 
+                              text=True,
+                              encoding='utf-8')  # 明确指定使用 UTF-8 编码
+        
+        if result.returncode == 0:
+            logging.info(f"脚本 {script_name} 运行成功")
+            if result.stdout:
+                logging.info(f"输出: {result.stdout}")
+        else:
+            logging.error(f"脚本 {script_name} 运行失败")
+            if result.stderr:
+                logging.error(f"错误: {result.stderr}")
+    except Exception as e:
+        logging.error(f"运行脚本 {script_name} 时发生错误: {str(e)}")
+
+
+def run_all_scripts():
+    """运行所有脚本"""
+    logging.info("开始执行所有脚本")
+    
+    # 获取当前脚本所在的目录
+    current_dir = os.path.dirname(os.path.abspath(__file__))
+    scripts = ['db2txt.py', 'usrSpareTime.py', 'compareDb2txt.py']
+    
+    for script in scripts:
+        # 使用完整的文件路径
+        script_path = os.path.join(current_dir, script)
+        if os.path.exists(script_path):
+            run_script(script_path)
+            time.sleep(5)
+        else:
+            logging.error(f"脚本文件不存在: {script_path}")
+    
+    logging.info("所有脚本执行完成")
+
+def main():
+    logging.info("启动定时任务程序")
+    
+    # 设置每小时运行一次
+    schedule.every().hour.at(":43").do(run_all_scripts)
+    
+    # 立即运行一次
+    run_all_scripts()
+    
+    # 持续运行
+    while True:
+        try:
+            schedule.run_pending()
+            time.sleep(60)
+        except Exception as e:
+            logging.error(f"运行时发生错误: {str(e)}")
+            time.sleep(60)  # 发生错误时等待一分钟后继续
+
+if __name__ == "__main__":
+    main() 

LLM/Notify/usrSpareTime.py

@@ -0,0 +1,148 @@
+# '''
+# Description: 
+# Author: Manda
+# Version: 
+# Date: 2025-03-30 16:42:47
+# LastEditors: mdhuang555 [email protected]
+# LastEditTime: 2025-03-30 16:59:19
+# '''
+import mysql.connector
+import os
+from datetime import datetime
+from collections import defaultdict
+from dataBaseConnecter import DatabaseConnector
+import sys
+import io
+sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
+
+def connect_to_database(db_config: dict) -> mysql.connector.MySQLConnection:
+    """连接到MySQL数据库"""
+    try:
+        conn = mysql.connector.connect(
+            host=db_config['host'],
+            user=db_config['user'],
+            password=db_config['password'],
+            database=db_config['database'],
+            charset='utf8mb4'
+        )
+        return conn
+    except Exception as e:
+        print(f"数据库连接错误: {e}")
+        return None
+
+def get_time_slot(hour: int, minute: int) -> str:
+    """将时间转换为40分钟一段的时间段"""
+    # 计算一天中的第几个40分钟
+    total_minutes = hour * 60 + minute
+    slot_index = total_minutes // 40
+    
+    # 计算时间段的起始和结束时间
+    start_minutes = slot_index * 40
+    end_minutes = start_minutes + 40
+    
+    start_hour = start_minutes // 60
+    start_minute = start_minutes % 60
+    end_hour = end_minutes // 60
+    end_minute = end_minutes % 60
+    
+    # 格式化时间段字符串
+    return f"{start_hour:02d}:{start_minute:02d}-{end_hour:02d}:{end_minute:02d}"
+
+def analyze_time_slots(db_connector: DatabaseConnector) -> dict:
+    """分析时间段分布"""
+    try:
+        # 连接数据库
+        conn = db_connector.connect_db()
+        if not conn:
+            print("无法连接到数据库")
+            return {}
+            
+        cursor = conn.cursor(dictionary=True)
+        
+        # 获取UCtodolist的数据和对应的ToDoList用户ID
+        query = """
+        SELECT uc.todo_id, uc.last_modified, t.user_id
+        FROM UCtodolist uc
+        JOIN ToDoList t ON uc.todo_id = t.todo_id
+        WHERE uc.last_modified IS NOT NULL
+        """
+        cursor.execute(query)
+        results = cursor.fetchall()
+        
+        # 按用户ID分组统计时间段
+        user_time_slots = defaultdict(lambda: defaultdict(int))
+        
+        for row in results:
+            if isinstance(row['last_modified'], datetime):
+                hour = row['last_modified'].hour
+                minute = row['last_modified'].minute
+                time_slot = get_time_slot(hour, minute)
+                user_time_slots[row['user_id']][time_slot] += 1
+        
+        return dict(user_time_slots)
+    
+    except Exception as e:
+        print(f"分析时间段时出错: {e}")
+        return {}
+    finally:
+        if 'cursor' in locals():
+            cursor.close()
+        if 'conn' in locals() and conn:
+            conn.close()
+
+def save_analysis_results(results: dict, output_dir: str = 'time_analysis'):
+    """保存分析结果到文件"""
+    if not os.path.exists(output_dir):
+        os.makedirs(output_dir)
+    
+    for user_id, time_slots in results.items():
+        filename = os.path.join(output_dir, f'user_{user_id}_time_analysis.txt')
+        
+        try:
+            with open(filename, 'w', encoding='utf-8') as f:
+                f.write(f"分析时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
+                f.write(f"用户ID: {user_id}\n")
+                f.write("=" * 50 + "\n\n")
+                
+                f.write("时间段使用频率统计（前6名）：\n")
+                # 按频率排序并获取前6个时段
+                top_slots = sorted(time_slots.items(), key=lambda x: x[1], reverse=True)[:6]
+                
+                for i, (slot, count) in enumerate(top_slots, 1):
+                    f.write(f"第{i}名: {slot}\n")
+                    f.write(f"  出现次数: {count}\n")
+                    percentage = (count / sum(time_slots.values())) * 100
+                    f.write(f"  占比: {percentage:.2f}%\n")
+                    f.write("-" * 30 + "\n")
+                
+                # 添加总计信息
+                f.write(f"\n总修改次数: {sum(time_slots.values())}\n")
+                f.write(f"总时间段数: {len(time_slots)}/36\n")
+                
+            print(f"已保存用户 {user_id} 的时间分析到文件: {filename}")
+            
+        except Exception as e:
+            print(f"保存用户 {user_id} 的分析结果时出错: {e}")
+
+def main():
+    print("正在连接数据库...")
+    
+    try:
+        # 创建数据库连接器实例
+        db_connector = DatabaseConnector()
+        
+        print("正在分析时间段分布...")
+        results = analyze_time_slots(db_connector)
+        
+        if results:
+            print(f"分析完成，共有 {len(results)} 个用户的数据")
+            save_analysis_results(results)
+            print("分析结果已保存到文件中")
+        else:
+            print("未找到可分析的数据")
+            
+    except Exception as e:
+        print(f"处理过程中出错: {e}")
+
+if __name__ == "__main__":
+    main()

LLM/filter_message/README.MD

@@ -0,0 +1,18 @@
+
+## 安装依赖
+
+
+```
+pip install -r requirements.txt
+```
+
+## 运行示例
+
+```
+python main.py
+```
+
+
+## 查看结果
+
+data目录下的json文件（未清洗 ）

LLM/filter_message/libs.py

@@ -0,0 +1,217 @@
+from typing import Optional
+
+import yaml
+import pymysql
+from openai import OpenAI
+
+def read_config(yaml_file):
+    """从yaml文件读取配置"""
+    with open(yaml_file, "r", encoding="utf-8") as f:
+        return yaml.safe_load(f)
+
+
+import mysql.connector
+import os
+from pathlib import Path
+
+
+def get_db_conn():
+    """获取复用数据库链接 (Azure MySQL) """
+    config = CONFIG["mysql"]
+
+    # 获取 SSL 证书路径
+    current_dir = Path(__file__).parent.absolute()
+    ssl_ca_path = current_dir / "DigiCertGlobalRootCA.crt.pem"
+
+    # 建立数据库连接
+    conn = mysql.connector.connect(
+        host=config["host"],
+        port=config.get("port", 3306),
+        user=config["user"],
+        password=config["password"],
+        database=config["database"],
+        ssl_ca=str(ssl_ca_path),
+        ssl_disabled=False
+    )
+
+    return conn
+
+
+def execute_sql(sql):
+    """执行sql"""
+    with DB_CONN.cursor() as cursor:
+        cursor.execute(sql)
+
+        # 判断是否为SELECT语句
+        if sql.strip().upper().startswith("SELECT"):
+            result = cursor.fetchall()
+            DB_CONN.commit()  # 提交事务，虽然SELECT语句不需要，但养成习惯
+            return result
+        else:
+            affected_rows = cursor.rowcount
+            DB_CONN.commit()  # 提交事务，INSERT/UPDATE/DELETE需要提交
+            return affected_rows
+
+
+def release():
+    """释放资源"""
+    DB_CONN.close()
+
+    del CONFIG
+    del DB_CONN
+
+
+def get_llm():
+    config = CONFIG["openai"]
+    client = OpenAI(base_url=config["base_url"], api_key=config["api_key"])
+
+    return client
+
+
+def send_llm(messages: list[dict[str, str]], model: Optional[str] = None, resp_json=False):
+    """调用LLM"""
+    print(">>>>>>>>>>>>>>>>>",messages)
+    config = CONFIG["openai"]
+
+    if model is None:
+        model = config["model"]
+
+    if resp_json:
+        completion = LLM.chat.completions.create(
+            model=model,  # 选择模型
+            messages=messages,
+            temperature=0,  # 为提高准确率，温度为0
+            response_format={ "type": "json_object" },
+        )
+    else:
+        completion = LLM.chat.completions.create(
+            model=model,  # 选择模型
+            messages=messages,
+            temperature=0,  # 为提高准确率，温度为0
+        )
+
+    print("<<<<<",completion.choices[0].message.content)
+    return completion.choices[0].message.content
+
+
+def send_llm_with_query(query):
+    messages = {
+        "role": "user",
+        "content": query,
+    },
+    return send_llm(messages)
+
+
+def send_llm_with_prompt(query):
+    system = """
+    # 角色
+    你是一个专业的短信内容分析助手，根据输入判断内容的类型及可信度，为用户使用信息提供依据和便利。
+
+    # 任务
+    对于输入的多条数据，分析每一条数据内容（主键：`message_id`）属于【物流取件、缴费充值、待付(还)款、会议邀约、其他】的可能性百分比。
+    主要对于聊天、问候、回执、结果通知、上月账单等信息不需要收件人进行下一步处理的信息，直接归到其他类进行忽略
+
+    # 要求
+    1. 以json格式输出
+    2. content简洁提炼关键词，字符数<20以内
+    3. 输入条数和输出条数完全一样
+    
+    # 输出示例
+    ```
+    [
+        {"message_id":"1111111","content":"账单805.57元待还","物流取件":0,"欠费缴纳":99,"待付(还)款":1: "会议邀约":0,"其他":0, "分类":"欠费缴纳"},
+        {"message_id":"222222","content":"邀请你加入飞书视频会议","物流取件":0,"欠费缴纳":0,"待付(还)款":1: "会议邀约":100,"其他":0, "分类":"会议"}
+    ]
+
+    ```
+        """
+
+    messages = [
+        {
+            "role": "system",
+            "content": system,
+        },
+        {
+            "role": "user",
+            "content": str(query),
+        }
+    ]
+    return send_llm(messages)
+
+def save_to_mysql(data):
+    """新增：保存数据到 MySQL"""
+    # 字段映射关系（中文键名 → 数据库英文列名）
+    COLUMN_MAPPING = {
+        "message_id": "message_id",
+        "content": "content",
+        "物流取件": "logistics_pickup",
+        "欠费缴纳": "overdue_payment",
+        "待付(还)款": "pending_payment",
+        "会议邀约": "meeting_invitation",
+        "其他": "other",
+        "分类": "category"
+    }
+
+    BATCH_SIZE = 100  # 每次插入 100 行，减少锁冲突
+    conn = get_db_conn()
+
+    try:
+        with conn.cursor() as cursor:
+            sql = f"""
+            INSERT INTO message_stats 
+            ({', '.join(COLUMN_MAPPING.values())})
+            VALUES ({', '.join(['%s'] * len(COLUMN_MAPPING))})
+            ON DUPLICATE KEY UPDATE
+            {', '.join([f"{col} = VALUES({col})" for col in COLUMN_MAPPING.values() if col != 'message_id'])}
+            """
+
+            values = []
+            for item in data:
+                item["content"] = str(item["content"]).encode('utf-8').decode('utf-8', errors='ignore')
+
+                # 规则 1: 会议 且 content 不包含 "邀请你加入飞书视频会议"，归类为 "其他"
+                # if item.get("分类") == "会议邀约" and "邀请你加入飞书视频会议" not in item.get("content", ""):
+                #     item["分类"] = "其他"
+
+                # 规则 2: 欠费缴纳 且 content 包含 "缴费支出"，归类为 "其他"
+                if item.get("分类") == "欠费缴纳" and "缴费支出" in item.get("content", ""):
+                    item["分类"] = "其他"
+
+                row = [item.get(key, None) for key in COLUMN_MAPPING.keys()]
+                values.append(row)
+
+            # 分批插入 message_stats
+            for i in range(0, len(values), BATCH_SIZE):
+                batch = values[i: i + BATCH_SIZE]
+                cursor.executemany(sql, batch)
+                conn.commit()
+                print(f"成功插入 {len(batch)} 条数据到 message_stats")
+
+            # **3. 插入 `filter_message` 表，仅插入分类不等于“其他”的数据**
+            filter_sql = """
+            INSERT IGNORE INTO filter_message (message_id, content)
+            VALUES (%s, %s)
+            """
+
+            filter_values = [
+                (item.get("message_id"), item.get("content")) for item in data if item.get("分类") != "其他"
+            ]
+
+            # 分批插入 filter_message
+            for i in range(0, len(filter_values), BATCH_SIZE):
+                batch = filter_values[i: i + BATCH_SIZE]
+                cursor.executemany(filter_sql, batch)
+                conn.commit()
+                print(f"成功插入 {len(batch)} 条数据到 filter_message")
+
+    except pymysql.MySQLError as e:
+        conn.rollback()
+        print(f"数据插入失败: {e}")
+
+    finally:
+        conn.close()
+###### init #####
+
+CONFIG = read_config("filter_llm_config.yaml")
+DB_CONN = get_db_conn()
+LLM = get_llm()

LLM/filter_message/main.py

@@ -0,0 +1,124 @@
+# main.py
+import json
+import time
+
+from libs import save_to_mysql, execute_sql, send_llm_with_prompt
+
+
+def get_message_with_page(page_num, page_size=10):
+    """分页查询数据库（保持不变）"""
+    offset = (page_num) * page_size
+    sql = f"""
+    SELECT 
+    m.content,
+    MAX(m.message_id) as message_id,
+    MAX(m.date) as date
+FROM 
+    Messages m
+    left join message_stats n on m.message_id=n.message_id
+WHERE 
+    m.app_name IN ('com.tencent.mm', 'SMS', 'com.ss.android.lark','com.ss.android.teams')
+    AND DATE(m.date) >= '2025-04-26'
+    AND LENGTH(m.content) > 50
+    AND m.content NOT LIKE '%可用余额%'
+    and n.message_id is null  
+GROUP BY 
+    m.content
+    LIMIT {page_size} OFFSET {offset};
+    """
+    print("======", sql)
+    return execute_sql(sql)
+
+
+def get_page_count(page_size=10): # 3，数据库分页数，该怎么自动计算；
+    sql = f"""
+        SELECT 
+    COUNT(1) AS total_rows
+FROM 
+    (
+        SELECT 
+            m.content
+        FROM 
+            Messages m
+            LEFT JOIN message_stats n ON m.message_id = n.message_id
+        WHERE 
+            m.app_name IN ('com.tencent.mm', 'SMS', 'com.ss.android.lark')
+            AND DATE(m.date) >= '2025-03-31'
+            AND LENGTH(m.content) > 50
+            AND m.content NOT LIKE '%可用余额%'
+            AND n.message_id IS NULL
+        GROUP BY 
+            m.content
+    ) AS subquery;
+        """
+    rows = execute_sql(sql)
+    count = rows[0][0]
+
+    if count % page_size == 0:
+        page_count = count // page_size
+    else:
+        page_count = (count // page_size) + 1
+
+    print(f'表中数据共 {page_count} 页')
+    return page_count
+
+
+def data_to_todo(data: tuple[tuple], todo_list: list[dict], debug=False): # 1，查漏补缺message id，保证输入message数=输出数，最终miss_rate=0
+    request_id_list = {int(d[1]) for d in data}
+
+    resp = send_llm_with_prompt(data)
+    print(f'请求id: {len(request_id_list)} ->{request_id_list}')
+
+    resp = resp.replace("```json", "").replace("```", "")
+    print('    ' + resp)
+    print('    ' + '-' * 20)
+
+    try:
+        parsed_resp = json.loads(resp)
+        response_id_list = {int(d['message_id']) for d in parsed_resp}
+        print(f'响应id: {len(response_id_list)} -> {response_id_list}')
+
+        diff = request_id_list - response_id_list
+        if diff:
+            print(f'本次处理有遗漏：{diff}')
+            diff_data = tuple(d for d in data if d[1] in diff)
+            data_to_todo(diff_data, todo_list)
+
+        todo_list.extend(parsed_resp)
+        print(f'{len(todo_list)=}')
+    except Exception as e:
+        print(f"解析响应失败: {e}")
+
+
+def main():
+    todo_list = []
+
+    page_count = get_page_count()
+
+    """ 从0开始算分页 """
+    for i in range(0, page_count):
+        print(f'正在处理第 {i + 1} / {page_count} 页数据')
+        data = get_message_with_page(i)
+        if not data:
+            print('没有更多数据了')
+            break
+
+        data_to_todo(data, todo_list)
+
+    # 保存到MySQL
+    if todo_list:
+        save_to_mysql(todo_list)
+        print(f"成功保存{len(todo_list)}条数据到数据库")
+    else:
+        print("没有需要保存的数据")
+
+
+def main_loop(): # 2，当表新增一条信息，自动化调用llm处理；
+    while True:
+        main()
+        time.sleep(30)
+
+
+if __name__ == '__main__':
+     main()
+    # main_loop()

LLM/filter_message/prompt.md

@@ -0,0 +1,96 @@
+## 人工标注数据
+
+| content                                                      | 是否正负样本 | 关键词                   | 类型     |
+| ------------------------------------------------------------ | ------------ | ------------------------ | -------- |
+| 【建设银行】您账户8699于3月24日12时22分向微信支付-扫二维码付款支出人民币15元,可用余额5433.72元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月24日12时11分向支付宝-天猫-深圳市升景科技有限公司支出人民币1889元,可用余额5465.6元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月24日12时11分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额5448.72元。 | FALSE        | 可用余额                 |          |
+| 【驿收发】您的京东包裹已到惜福堂对面快递驿站，请21:00前凭2-0147来取，详询16675200600 | TRUE         | 驿站、包裹、取件         | 包裹物流 |
+| 【中国农业银行】为您特别准备了一份专属月度账单，登掌银搜“月度账单”或戳  go.abchina.com/k/CS7 查收。拒收请回复R | FALSE        | 拒收                     |          |
+| 【中国农业银行】充话费最高减20元，本月登掌银【城市专区-优惠立减】或点  go.abchina.com/k/C05 先到先得。拒收请回复R | FALSE        | 拒收                     |          |
+| 刻刻:  【火山引擎】亲爱的用户，您有千万DeepSeek模型额度待领取哦！每邀请1位新用户注册和使用，最高可获得130元代金券，多邀多得上不封顶！点击下载海报，分享到更多渠道：szacq.cn/ewyeb/  拒收请回复R | FALSE        | 拒收                     |          |
+| M丶D:  【阿里云推广】尊敬的mandahuang，老友季钜惠！云服务器  99元/年，新购续费同享！戳>>https://t.aliyun.com/TIFqsCZY 立即抢购！ 拒收请回复R | FALSE        | 拒收                     |          |
+| 王斯煜[表情]Vince  黑客松nv: 【AlipayHK】於3月19日 08:15需要通過Apple 服務(Apple services)待付款支付HKD8.00 | TRUE         | 账单、待付款             | 支付     |
+| [2条]李JK老师-1228:  【百度】亲爱的开发者您好，文心智能体年底送福利，即日起至12月30日，创建并提交您的智能体，即有机会获得现金卡福利，最高500元，多档奖励，中奖机会大！登陆天数越多，创建有创意，中奖率越高！福利倒计时！点击参与！https://agents.baidu.com/activity/detail/13 | FALSE        | 送福利、有机会           |          |
+| [5条]LJK86:  【百度】亲爱的开发者您好，文心智能体年底送福利，即日起至12月30日，创建并提交您的智能体，即有机会获得现金卡福利，最高500元，多档奖励，中奖机会大！登陆天数越多，创建有创意，中奖率越高！福利倒计时！点击参与！https://agents.baidu.com/activity/detail/13 | FALSE        | 送福利、有机会           |          |
+| [3条]M丶D:  【韵达快递】亲434466097408983超10小时未取出，如需帮助或有问题请致电15900077340 、020-89725127 | TRUE         | 快递、取件               | 包裹物流 |
+| [2条]LJK86:  【智谱AI】亲爱的开发者您好，感谢您参与智谱开放平台满意度调研，您的智谱清言月卡奖励正在发放，请在链接中输入您问卷中填写的手机号查询礼品码，前往PC/APP智谱清言会员充值页，选择“礼品码兑换”。查询链接：https://zhipu-ai.feishu.cn/share/base/query/shr... | FALSE        | 满意度调研               |          |
+| [2条]M丶D:  【中国电信】流量满满，温暖相伴，我们特别为你准备了预存领10GB的流量大礼，无论是与家人视频通话，还是朋友间分享趣事，都能畅通无阻，让爱不断线。马上戳  https://vipxjzl.mini189.cn/BG/ 了解吧，具体以实际页面展示为准，如已办理请忽略，转发无效。拒收请回复R | FALSE        | 拒收                     |          |
+| [3条]王斯煜[表情]Vince  黑客松nv: 【中通快递】包裹已到深圳光明正大城商业街103号店，取件码3-5-1092。到店扫“取件二维码”，线上查询更方便！询19128399078 | TRUE         | 快递、包裹、取件码       | 包裹物流 |
+| 王斯煜[表情]Vince  黑客松nv:  尊敬的*斯煜，您在我行办理的1笔个人贷款需于2024年12月08日17:00前还款，当期还款金额本息合计999999.84元，请您留意尾号0455的账户可用余额是否充足，避免因贷款逾期影响个人征信。具体贷款信息可通过工行网上银行、手机银行或致电贷款经办行查询。【工商银行】 | TRUE         | 银行、贷款、还款         | 支付     |
+| [2条]李JK老师-1228:  【51CTO学堂】《DeepSeek训练营》火爆来袭！0元报名，学习AI核心技术，解锁职业新可能 zt60.cn/CMPE9 拒收请回复R | FALSE        | 拒收                     |          |
+| [3条]刻刻:  60GB本地數據及5000分鐘本地通話30日組合成功開啟，餘額已被扣除，有效期至10/02/2025 23:59。請立即登入MyLink App  bit.ly/MySimMyLink  進行增值，若賬戶餘額充足，60GB本地數據及5000分鐘本地通話30日組合將每30日以$38自動續期，並於餘額內自動... | FALSE        |                          |          |
+| [3条]李JK老师-1228:  【湖南通信】尊敬的用户：您订购的卡已配送暂未签收，为了更好的为您服务，烦请回复：未拿到卡且需要回复1，卡在站点未取件回复2，已取件回复3。如有疑问请拨打4008155555 | TRUE         | 配送、未签收、未取件     | 包裹物流 |
+| [2条]李JK老师-1228:  【深势科技】Bohrium用户您好，您有共计余额20.00元的体验卡将于2024-12-24 23:59:59到期，请及时使用。 | FALSE        | 共计余额                 |          |
+| [3条]李JK老师-1228:  【讯飞开放平台】到期预警！尊敬的会员用户：您的个人级乐享会员，将于2024-12-08  00:00:00正式到期,截至目前仅剩7天。如您要继续使用，自即日起，7天内完成会员体验问卷http://1024-2019.iflytek.com/h5/vip-ques?t=2024-12-15，即可免费续约... | TRUE         | 到期                     | 到期提醒 |
+| M丶D:  【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。 | TRUE         | 快递、取件               | 包裹物流 |
+| [33条]斯煜[表情]Vince:  【中通快递】包裹已到深圳光明正大城商业街103号店，取件码3-5-1092。到店扫“取件二维码”，线上查询更方便！询19128399078 | TRUE         | 快递、取件、取件码       | 包裹物流 |
+| 王斯煜[表情]Vince  黑客松nv: 【申通快递】快递尾号7117已在代收点1天未取，请尽快取件，详询13434525312，最新快递状态请访问t.sto.cn/18PBt4 | TRUE         | 快递、取件               | 包裹物流 |
+| [2条]王斯煜[表情]Vince  黑客松nv: 【申通快递】包裹已到深圳光明正大城商业街103号店，取件码5-5-7117。到店扫“取件二维码”，线上查询更方便！询19128399078 | TRUE         | 快递、取件码             | 包裹物流 |
+| [2条]王斯煜[表情]Vince  黑客松nv: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba | TRUE         | 账单、还款               | 支付     |
+| [32条]斯煜[表情]Vince:  【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba | TRUE         | 账单、还款               | 支付     |
+| [30条]斯煜[表情]Vince:  【美团月付】您的2月账单805.57元需要付款，查看详情点击 dpurl.cn/80JQdKHa | TRUE         | 账单、付款               | 支付     |
+| [26条]斯煜[表情]Vince:  Mox: 你的Mox Credit月結單已準備就緒。你可登入Mox應用程式查看。 | TRUE         | 账单                     | 支付     |
+| 【建设银行】您账户8699于3月22日9时51分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7414.6元。 | FALSE        | 可用余额                 |          |
+| [3条]M丶D:  明天九点开会                                     | TRUE         | 时间、开会               | 会议     |
+| 【建设银行】您账户8699于3月22日9时42分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7431.48元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月22日9时32分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7448.36元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月22日9时12分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7465.24元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月22日8时53分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7482.12元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月22日18时2分向微信支付-羊城通缴费支出人民币2元,可用余额7397.6元。 | FALSE        | 可用余额                 |          |
+| 【招商银行】您的风险评估已到期，为避免错过我行优质产品信息，请及时重评！点击  cmbt.cn/a/zhV 去评估。如已完成或销户请忽略 | FALSE        | 请忽略                   |          |
+| 【建设银行】您账户8699于3月22日23时45分向微信支付-西苑出版社支出人民币180元,可用余额7215.6元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月22日22时47分向微信支付-羊城通缴费支出人民币2���,可用余额7395.6元。 | FALSE        | 可用余额                 |          |
+| 【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。 | TRUE         | 快递、丰巢、取件         | 包裹物流 |
+| 【建设银行】您账户8699于3月22日13时16分向微信支付-扫二维码付款支出人民币15元,可用余额7399.6元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月21日8时11分向微信支付-小霞包点（香雪店）支出人民币5.5元,可用余额7566.14元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月21日8时7分向微信支付-羊城通缴费支出人民币3.5元,可用余额7571.64元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月21日20时5分向微信支付-羊城通缴费支出人民币1元,可用余额7519.38元。 | FALSE        | 可用余额                 |          |
+| 【中国农业银行】百万立减金限时抢，最高10元立减金等您拿，本月戳  go.abchina.com/k/CfK 直达，先到先得。拒收请回复R | FALSE        | 拒收                     |          |
+| 【建设银行】您账户8699于3月21日15时28分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7520.38元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月21日15时28分向支付宝-理财-天弘基金管理有限公司支出人民币16.88元,可用余额7537.26元。 | FALSE        | 可用余额                 |          |
+| M丶D:  【话费账单】尊敬的150****9503客户，您02月01日- 02月28日共消费8.00元。主要消费项目包括： | FALSE        | 共消费                   |          |
+| 菜鳥包裹HK00083544922到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1002，3個工作日内取。 | TRUE         | 菜鸟、包裹、到达、提件码 | 包裹物流 |
+| 您的集運單SF3148634434778因其他异常已拒收退回賣家，建議您聯系賣家處理，如有必要可申請退款 | TRUE         | 申请退款                 | 包裹物流 |
+| 菜鳥包裹HK00083525678到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1004，3個工作日内取。 | TRUE         | 菜鸟、包裹、到达、提件码 | 包裹物流 |
+| [10条]王斯煜[表情]Vince  黑客松nv: 菜鳥包裹HK00083544922到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1002，3個工作日内取。 | TRUE         | 菜鸟、包裹、到达、提件码 | 包裹物流 |
+| [11条]王斯煜[表情]Vince  黑客松nv: 您的集運單SF3148634434778因其他异常已拒收退回賣家，建議您聯系賣家處理，如有必要可申請退款 | TRUE         | 申请退款                 | 包裹物流 |
+| 【建设银行】您账户8699于3月21日12时20分向微信支付-扫二维码付款支出人民币12元,可用余额7554.14元。 | FALSE        | 可用余额                 |          |
+| [2条]AlisaGG陈G老师1010:  预约一个会和你 15分钟。我也正经了[加油] | TRUE         | 预约、开会、时间         | 会议     |
+| 【建设银行】您账户8699于3月20日20时3分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7574.12元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月20日19时51分向微信支付-羊城通缴费支出人民币3.5元,可用余额7591元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月20日18时36分向微信支付-羊城通缴费支出人民币1元,可用余额7594.5元。 | FALSE        | 可用余额                 |          |
+| WANG  Siyu: @AlisaGG 6：00PM后 今晚 什么时候有空             | TRUE         | 有空、时间               | 会议     |
+| 【建设银行】您账户8699于3月27日8时3分向微信支付-羊城通缴费支出人民币3.5元,可用余额5240.93元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月27日8时28分向微信支付-百度平台商家支出人民币18.95元,可用余额5221.98元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月27日9时12分向支付宝-理财-天弘基金管理有限公司支出人民币16.88元,可用余额5205.1元。 | FALSE        | 可用余额                 |          |
+| 【广州银行】您尾号5138的卡片转入人民币550.00元，本期账单已还清。绑定广州银行信用卡官微实时查账。 | FALSE        | 账单、还清               |          |
+| 【建设银行】您账户8699于3月27日9时26分向支付宝-黄敏达还款支出人民币550元,可用余额4655.1元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月27日9时32分向��付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额4620.64元。 | FALSE        | 可用余额                 |          |
+| 【建设银行】您账户8699于3月27日9时48分向微信支付-壹伴助手支出人民币399元,可用余额4221.64元。 | FALSE        | 可用余额                 |          |
+| 【广州银行】黄敏达先生，您好。您的信用卡欠款未缴，请即日内还款，否则我行不排除根据客户协议委托第三方公司向您催缴。若已还款无需理会 | TRUE         | 信用卡、还款             | 支付     |
+| 【中国农业银行】尊敬的用户，您已超过1个月未登录账户，邀您点击  go.abchina.com/k/C7C 查看账户详情。拒收请回复R | FALSE        | 拒收                     |          |
+| 【建设银行】您账户8699于3月27日12时38分向微信支付-兰州拉面（佳大）支出人民币16元,可用余额4205.64元。 | FALSE        | 可用余额                 |          |
+| [23条]M丶D: 1000本链接: https://pan.baidu.com/s/13dV3m54iGE8oWgYtkARQPw?pwd=vwq3 提取码: vwq3 复制这段内容后打开百度网盘手机App，操作更方便哦  | FALSE        |    百度网盘              |          |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE        | 可用余额                 |          |
+| 招商银行风险评估到期提醒 | FALSE        | 到期提醒                 |     提醒     |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE        | 可用余额                 |          |
+| 重构工作完成时间                                                      | FALSE |                    | 聊天     |
+| 上课时间 | FALSE        |                  |    提醒     |
+| 明天上班                                                      | FALSE |                    | 聊天     |
+| 等待黄老师远程会议 | FALSE        |       等待           |    聊天     |
+| 确认代码与数据库兼容性                                                      | FALSE |          确认          |     |
+| 补全项目说明书 | FALSE        |                  |         |
+| 更新readme文档                                                      | FALSE |                    | 待办    |
+| 确认是否需要关防火墙 | FALSE        |                 |         |
+| 使用企业邮箱                                                      | FALSE |                    |     |
+| 处理手机验证问题 | FALSE        |                  |         |
+| 分享达子的课程到飞书共享空间                                                      | FALSE |                    |     |
+| 明天下午3点有会议 | TRUE        |         会议         |    开会     |
+| 确认会议时间                                                      | FALSE |                    |  细节讨论   |
+| 把代码上传到云上的自己的分支                                                      | FALSE |                    |  讨论   |
+| "大数据需求准入评审周会，时间：11:00 -12:30 参会人：所有 地点：2栋17楼-用户-配有飞书会议-优先5人以上预定(12)深圳新一个代2栋"    | TRUE |        参会人、时间、地点、会议        |  开会   |
+| [26条]斯煜[表情]Vince: Mox: 你的Mox Credit月結單已準備就緒。你可登入Mox應用程式查看。  | TRUE |     账单      |  缴费   |
+| 飞书验证码、处理飞书邮箱问题                                                     | FALSE |                    | 无效信息    |
+| Python版本确认                                                     | FALSE |                    | 无效信息    |

LLM/filter_message/prompt.txt

@@ -0,0 +1,90 @@
+## 人工标注数据
+
+| content | 是否正负样本 | 关键词 | 类型 |
+| --- | --- | --- | --- |
+| 【建设银行】您账户8699于3月26日8时6分向微信支付-羊城通缴费支出人民币3.5元,可用余额5285.33元。 | FALSE | 余额 | 提醒 |
+| 【建设银行】您账户8699于3月26日8时11分向微信支付-小霞包点（香雪店）支出人民币5.5元,可用余额5279.83元。 | FALSE | 余额 | 提醒 |
+| [3条]LJK86: 您的号卡已在配送途中，物流单号SF3147624215612,点击：http://t.hn.189.cn/EZv6bazD  ，查询物流进度。如已签收请在有效期内先实名激活。客服热线4008155555【中国电信】 | FALSE |  |  |
+| AlisaGG陈G老师1010: 【招商银行】您本月办理分期还款可享限时2.3折优惠！打开掌上生活APP，搜“分期还款”立享折扣优惠！资格实时测评，拒收请回复R | FALSE |  |  |
+| "王斯煜[表情]Vince 黑客松nv: @AlisaGG @LJK86 明天中午再对 WANG Siyu邀请你加入飞书视频会议<br>会议主题：FilterLLM ToDoGenLLM PR Merge讨论<br>会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)<br>会议 ID：656 445 907<br>会议链接：https://..." | TRUE | 飞书视频会议 | 会议 |
+| [6条]王斯煜[表情]Vince 黑客松nv: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se... | FALSE |  |  |
+| [4条]斯煜[表情]Vince: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba | TRUE | 还款 | 待付(还)款 |
+| [4条]斯煜[表情]Vince: 【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。 | TRUE | 快递、取件 | 物流取件 |
+| [6条]斯煜[表情]Vince: 尊敬的*斯煜，您在我行办理的1笔个人贷款需于2024年12月08日17:00前还款，当期还款金额本息合计999999.84元，请您留意尾号0455的账户可用余额是否充足，避免因贷款逾期影响个人征信。具体贷款信息可通过工行网上银行、手机银行或致电贷款经办行查询。【工商银行】 | TRUE | 还款 | 待付(还)款 |
+| [6条]斯煜[表情]Vince: Your Mint 3-Month, Unlimited plan expires in 2 days. Log into your account at my.mintmobile.com or via our app to make a payment and keep your Mint se... | FALSE |  | 提醒 |
+| AlisaGG陈G老师1010: 【驿收发】您的邮政包裹已到凯丰花园2栋驿站，请23:00前凭5-5-6530来取，详询18320926368 | TRUE |  | 物流取件 |
+| [2条]AlisaGG: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。 | TRUE | 取件码 | 物流取件 |
+| [2条]AlisaGG陈G老师1010: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。 | TRUE | 取件码 | 物流取件 |
+| "[3条]AlisaGG陈G老师1010: 取件通知<br>取件码: 11724837<br>运单号: 464285154986072<br>取件地址: 深圳新一代产业园P2出入口内侧4号丰巢柜<br>配送人员: 18124519013<br>计费规则: 查看详情" | TRUE | 取件码、运单号 | 物流取件 |
+| "[4条]AlisaGG陈G老师1010: 取件再次提醒<br>取件码：: 27696201<br>配送公司：: 申通快递<br>运单号：: 777293635831671<br>配送员手机：: 13392809673<br>取件地址：: 深圳新一代产业园P2出入口内侧4号丰巢柜" | TRUE | 取件码、运单号 | 物流取件 |
+| [5条]李JK老师-1228: 【菜鸟驿站】请凭140-3-1019到菜鸟驿站取件，查询详情u.cainiao.com/53h4bSr7zrh | TRUE | 取件码、菜鸟驿站 | 物流取件 |
+| "斯煜[表情]Vince: WANG Siyu邀请你加入飞书视频会议<br>会议主题：FilterLLM ToDoGenLLM PR Merge讨论<br>会议时间：3月31日 (今天) 12:00 - 12:30 (GMT+8)<br>会议 ID：656 445 907<br>会议链接：https://vc.feishu.cn/j/65644590  ..." | TRUE | 飞书视频会议 | 会议 |
+| 刻刻: 【停机前提醒】尊敬的移动客户，您好！您的账户余额不足被限制使用。现提醒您需充值缴费至少32.49元，以确保您继续享受畅通的通信服务。诚邀您一键办理自动充服务，自动充值缴费更轻松：https://dx.10086.cn/7WyRLA  。 心级服务、让爱连接【中国移动】 | TRUE | 缴费、余额不足 | 待付(还)款 |
+| AlisaGG: 【小象超市】您的商品已放置在门口，因有易碎等商品请尽快取回，如有疑问请联系15794935204 。祝您生活愉快！ | FALSE |  | 提醒 |
+| 【美团月付】您4月账单8937.95元待还，最后还款日为本月8号，查账或立即还款点 dpurl.cn/kVln1cqa | TRUE | 还款 | 待付(还)款 |
+| 建设银行】您账户8699于3月24日12��22分向微信支付-扫二维码付款支出人民币15元,可用余额5433.72元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月24日12时11分向支付宝-天猫-深圳市升景科技有限公司支出人民币1889元,可用余额5465.6元。 | FALSE | 可用余额 |  |
+| 【建设银行】您账户8699于3月24日12时11分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额5448.72元。 | FALSE | 可用余额 |  |
+| 【驿收发】您的京东包裹已到惜福堂对面快递驿站，请21:00前凭2-0147来取，详询16675200600 | TRUE | 驿站、包裹、取件 | 物流取件 |
+| 【中国农业银行】为您特别准备了一份专属月度账单，登掌银搜“月度账单”或戳 go.abchina.com/k/CS7 查收。拒收请回复R | FALSE | 拒收 | 提醒 |
+| 刻刻: 【火山引擎】亲爱的用户，您有千万DeepSeek模型额度待领取哦！每邀请1位新用户注册和使用，最高可获得130元代金券，多邀多得上不封顶！点击下载海报，分享到更多渠道：szacq.cn/ewyeb/ 拒收请回复R | FALSE | 拒收 | 提醒 |
+| M丶D: 【阿里云推广】尊敬的mandahuang，老友季钜惠！云服务器 99元/年，新购续费同享！戳>><url id="cvlp38lgsoaarffpph8g" type="url" status="parsed" title="阿里云权益中心" wc="3700">https://t.aliyun.com/TIFqsCZY  </url> 立即抢购！拒收请回复R | FALSE | 拒收 | 提醒 |
+| 【AlipayHK】於3月19日 08:15需要通過Apple 服務(Apple services)待付款支付HKD8.00 | TRUE | 账单、待付款 | 支付 |
+| [2条]李JK老师-1228: 【百度】亲爱的开发者您好，文心智能体年底送福利，即日起至12月30日，创建并提交您的智能体，即有机会获得现金卡福利，最高500元，多档奖励，中奖机会大！登陆天数越多，创建有创意，中奖率越高！福利倒计时！点击参与！<url id="cvlp38lgsoaarffpph9g" type="url" status="failed" title="" wc="0">https://agents.baidu.com/activity/detail/13  </url> | FALSE | 送福利、有机会 | 无效信息 |
+| [5条]LJK86: 【百度】亲爱的开发者您好，文心智能体年底送福利，即日起至12月30日，创建并提交您的智能体，即有机会获得现金卡福利，最高500元，多档奖励，中奖机会大！登陆天数越多，创建有创意，中奖率越高！福利倒计时！点击参与！<url id="cvlp38lgsoaarffpph9g" type="url" status="failed" title="" wc="0">https://agents.baidu.com/activity/detail/13  </url> | FALSE | 送福利、有机会 | 无效信息 |
+| [3条]M丶D: 【韵达快递】亲434466097408983超10小时未取出，如需帮助或有问题请致电15900077340 、020-89725127 | TRUE | 取件 | 物流取件 |
+| [2条]LJK86: 【智谱AI】亲爱的开发者您好，感谢您参与智谱开放平台满意度调研，您的智谱清言月卡奖励正在发放，请在链接中输入您问卷中填写的手机号查询礼品码，前往PC/APP智谱清言会员充值页，选择“礼品码兑换”。查询链接：<url id="cvlp38lgsoaarffppha0" type="url" status="parsed" title="Feishu - Log in" wc="339">https://zhipu-ai.feishu.cn/share/base/query/shr  </url> ... | FALSE | 满意度调研 | 无效信息 |
+| [2条]M丶D: 【中国电信】流量满满，温暖相伴，我们特别为你准备了预存领10GB的流量大礼，无论是与家人视频通话，还是朋友间分享趣事，都能畅通无阻，让爱不断线。马上戳 <url id="cvlp38lgsoaarffpphag" type="url" status="parsed" title="预存领流量" wc="3306">https://vipxjzl.mini189.cn/BG/  </url> 了解吧，具体以实际页面展示为准，如已办理请忽略，转发无效。拒收请回复R | FALSE | 拒收 |  |
+| [3条]王斯煜[表情]Vince 黑客松nv: 【中通快递】包裹已到深圳光明正大城商业街103号店，取件码3-5-1092。到店扫“取件二维码”，线上查询更方便！询19128399078 | TRUE | 快递、包裹、取件码 | 物流取件 |
+| [2条]李JK老师-1228: 【51CTO学堂】《DeepSeek训练营》火爆来袭！0元报名，学习AI核心技术，解锁职业新可能 zt60.cn/CMPE9 拒收请回复R | FALSE | 拒收 | 无效信息 |
+| [3条]刻刻: 60GB本地數據及5000分鐘本地通話30日組合成功開啟，餘額已被扣除，有效期至10/02/2025 23:59。請立即登入MyLink App bit.ly/MySimMyLink 進行增值，若賬戶餘額充足，60GB本地數據及5000分鐘本地通話30日組合將每30日以$38自動續期，並於餘額內自動... | FALSE |  | 提醒 |
+| [3条]李JK老师-1228: 【湖南通信】尊敬的用户：您订购的卡已配送暂未签收，为了更好的为您服务，烦请回复：未拿到卡且需要回复1，卡在站点未取件回复2，已取件回复3。如有疑问请拨打4008155555 | TRUE | 未签收 | 包裹物流 |
+| [2条]李JK老师-1228: 【深势科技】Bohrium用户您好，您有共计余额20.00元的体验卡将于2024-12-24 23:59:59到期，请及时使用。 | FALSE | 共计余额 | 提醒 |
+| [3条]李JK老师-1228: 【讯飞开放平台】到期预警！尊敬的会员用户：您的个人级乐享会员，将于2024-12-08 00:00:00正式到期,截至目前仅剩7天。如您要继续使用，自即日起，7天内完成会员体验问卷<url id="cvlp38lgsoaarffpphb0" type="url" status="parsed" title="乐享会员调查问卷" wc="1540">http://1024-2019.iflytek.com/h5/vip-ques?t=2024-12-15  </url> ，即可免费续约... | FALSE | 到期 | 提醒 |
+| M中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。 | TRUE | 丰巢、取件 | 物流取件 |
+| [33条]斯煜[表情]Vince: 【中通快递】包裹已到深圳光明正大城商业街103号店，取件码3-5-1092。到店扫“取件二维码”，线上查询更方便！询19128399078 | TRUE |  | 物流取件 |
+| 王斯煜[表情]Vince 黑客松nv: 【申通快递】快递尾号7117已在代收点1天未取，请尽快取件，详询13434525312，最新快递状态请访问t.sto.cn/18PBt4 | TRUE |  | 物流取件 |
+| [2条]王斯煜[表情]Vince 黑客松nv: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba | TRUE | 账单、还款 | 物流取件 |
+| [32条]斯煜[表情]Vince: 【美团月付】您2月账单805.57元待还，最后还款日为本月22号，查账或立即还款点 dpurl.cn/HXFUxEba | TRUE | 账单、还款 | 待付(还)款 |
+| [26条]斯煜[表情]Vince: Mox: 你的Mox Credit月結單已準備就緒。你可登入Mox應用程式查看。 | TRUE | 账单 | 待付(还)款 |
+| 【建设银行】您账户8699于3月22日9时51分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7414.6元。 | FALSE | 可用余额 | 提醒 |
+| [3条]M丶D: 明天九点开会 | TRUE | 时间、开会 | 会议 |
+| 【建设银行】您账户8699于3月22日9时42分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7431.48元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月22日18时2分向微信支付-羊城通缴费支出人民币2元,可用余额7397.6元。 | FALSE | 可用余额 |  |
+| 【招商银行】您的风险评估已到期，为避免错过我行优质产品信息，请及时重评！点击 cmbt.cn/a/zhV 去评估。如已完成或销户请忽略 | FALSE | 风险评估 | 提醒 |
+| 【建设银行】您账户8699于3月22日23时45分向微信支付-西苑出版社支出人民币180元,可用余额7215.6元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月22日22时47分向微信支付-羊城通缴费支出人民币2元,可用余额7395.6元。 | FALSE | 可用余额 | 提醒 |
+| 【中通快递】73547529665397放在丰巢的包裹请及时取件，如有问题致电15900077340、020-22504077。 | TRUE | 快递、丰巢、取件 | 包裹物流 |
+| 【建设银行】您账户8699于3月22日13时16分向微信支付-扫二维码付款支出人民币15元,可用余额7399.6元。 | FALSE | 可用余额 |  |
+| 【建设银行】您账户8699于3月21日8时11分向微信支付-小霞包点（香雪店）支出人民币5.5元,可用余额7566.14元。 | FALSE | 可用余额 |  |
+| 【建设银行】您账户8699于3月21日8时7分向微信支付-羊城通缴费支出人民币3.5元,可用余额7571.64元。 | FALSE | 可用余额 |  |
+| 【建设银行】您账户8699于3月21日20时5分向微信支付-羊城通缴费支出人民币1元,可用余额7519.38元。 | FALSE | 可用余额 |  |
+| 【中国农业银行】百万立减金限时抢，最高10元立减金等您拿，本月戳 go.abchina.com/k/CfK 直达，先到先得。拒收请回复R | FALSE | 拒收 | 提醒 |
+| 【建设银行】您账户8699于3月21日15时28分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额7520.38元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月21日15时28分向支付宝-理财-天弘基金管理有限公司支出人民币16.88元,可用余额7537.26元。 | FALSE | 可用余额 | 提醒 |
+| M丶D: 【话费账单】尊敬的150****9503客户，您02月01日- 02月28日共消费8.00元。主要消费项目包括： | FALSE | 共消费 | 提醒 |
+| 菜鳥包裹HK00083544922到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1002，3個工作日内取。 | TRUE |  | 物流取件 |
+| 您的集運單SF3148634434778因其他异常已拒收退回賣家，建議您聯系賣家處理，如有必要可申請退款 | FALSE |  | 提醒 |
+| 菜鳥包裹HK00083525678到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1004，3個工作日内取。 | TRUE |  | 物流取件 |
+| [10条]王斯煜[表情]Vince 黑客松nv: 菜鳥包裹HK00083544922到達香港仔置富道19號置富花園19座地下B舖，提件碼7-1-1002，3個工作日内取。 | TRUE |  | 物流取件 |
+| 【建设银行】您账户8699于3月21日12时20分向微信支付-扫二维码付款支出人民币12元,可用余额7554.14元。 | FALSE | 可用余额 |  |
+| [2条]AlisaGG陈G老师1010: 预约一个会和你 15分钟。我也正经了[加油] | TRUE |  | 会议 |
+| 【建设银行】您账户8699于3月27日8时3分向微信支付-羊城通缴费支出人民币3.5元,可用余额5240.93元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月27日8时28分向微信支付-百度平台商家支出人民币18.95元,可用余额5221.98元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时12分向支付宝-理财-天弘基金管理有限公司支出人民币16.88元,可用余额5205.1元。 | FALSE | 可用余额 | 提醒 |
+| 【广州银行】您尾号5138的卡片转入人民币550.00元，本期账单已还清。绑定广州银行信用卡官微实时查账。 | FALSE | 账单、还清 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时26分向支付宝-黄敏达还款支出人民币550元,可用余额4655.1元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-天弘基金管理有限公司支出人民币16.88元,可用余额4620.64元。 | FALSE | 可用余额 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时48分向微信支付-壹伴助手支出人民币399元,可用余额4221.64元。 | FALSE | 可用余额 | 提醒 |
+| 【广州银行】黄敏达先生，您好。您的信用卡欠款未缴，请即日内还款，否则我行不排除根据客户协议委托第三方公司向您催缴。若已还款无需理会 | TRUE | 信用卡、还款 | 待付(还)款 |
+| 【中国农业银行】尊敬的用户，您已超过1个月未登录账户，邀您点击 go.abchina.com/k/C7C 查看账户详情。拒收请回复R | FALSE | 拒收 | 提醒 |
+| 【建设银行】您账户8699于3月27日12时38分向微信支付-兰州拉面（佳大）支出人民币16元,可用余额4205.64元。 | FALSE | 可用余额 | 提醒 |
+| [23条]M丶D: 1000本链接: <url id="cvlp38lgsoaarffpphbg" type="url" status="failed" title="" wc="0">https://pan.baidu.com/s/13dV3m54iGE8oWgYtkARQPw?pwd=vwq3  </url> 提取码: vwq3 复制这段内容后打开百度网盘手机App，操作更方便哦 | FALSE | 百度网盘 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE | 可用余额 | 提醒 |
+| 招商银行风险评估到期提醒 | FALSE | 到期提醒 | 提醒 |
+| 【建设银行】您账户8699于3月27日9时32分向支付宝-广州市自来水有限公司支出人民币17.58元,可用余额4637.52元。 | FALSE | 可用余额 | 提醒 |
+| 明天下午3点有会议 | TRUE |  | 会议 |
+| 确认会议时间 | FALSE |  | 无效信息 |
+| 把代码上传到云上的自己的分支 | FALSE |  | 无效信息 |
+| "大数据需求准入评审周会，时间：11:00 -12:30 参会人：所有 地点：2栋17楼-用户-配有飞书会议-优先5人以上预定(12)深圳新一个代2栋" | TRUE | 飞书会议 | 会议 |

LLM/filter_message/requirements.txt

@@ -0,0 +1,4 @@
+openai
+pyyaml
+pymysql
+black

LLM/orchestrator.py

@@ -0,0 +1,95 @@
+import time
+import yaml
+import threading
+import importlib
+from pathlib import Path
+import sys
+# import yaml # Removed yaml import
+import mysql.connector as pymysql # Use mysql-connector-python alias
+
+# Import config loader from todogen_LLM
+from todogen_LLM.config_loader import get_mysql_config
+
+# Removed local load_config function and CONFIG/MYSQL_CONFIG globals
+# # 配置
+def load_config():
+     config_path = Path(__file__).parent / "todogen_LLM" / "todogen_LLM_config.yaml"
+     with open(config_path, 'r', encoding='utf-8') as f:
+         return yaml.safe_load(f)
+ 
+CONFIG = load_config()
+MYSQL_CONFIG = CONFIG['mysql']
+
+# 数据库
+def get_db_conn():
+    mysql_config = get_mysql_config()
+    return pymysql.connect(
+        host=mysql_config['host'],
+        port=mysql_config.get('port', 3306),
+        user=mysql_config['user'],
+        password=mysql_config['password'],
+        database=mysql_config['database'],
+        ssl_ca=mysql_config['ssl_ca'], # path 
+        ssl_disabled=False, #   enabled
+        charset='utf8mb4',
+        autocommit=True
+    )
+
+def get_latest_update_time(conn):
+    with conn.cursor() as cursor:
+        cursor.execute("SELECT MAX(date) FROM Messages")
+        result = cursor.fetchone()
+        return result[0]
+
+#filter_llm主入口
+def run_filter_llm():
+    sys.path.append(str(Path(__file__).parent / 'filter_llm'))
+    main = importlib.import_module('main')
+    if hasattr(main, 'main'):
+        main.main()
+    else:
+        raise RuntimeError('filter_llm.main.py未找到main函数')
+
+#todogen_LLM主入口
+def run_todogen_llm():
+    sys.path.append(str(Path(__file__).parent / 'todogen_LLM'))
+    todogen = importlib.import_module('todogen_llm')
+    if hasattr(todogen, 'main'):
+        todogen.main()
+    else:
+        if hasattr(todogen, 'load_formatted_data') and hasattr(todogen, 'process_data'):
+            data = todogen.load_formatted_data()
+            todogen.process_data(data)
+        else:
+            raise RuntimeError('todogen_llm.py未找到main或核心处理函数')
+
+# 调用Notify主入口
+#// def run_notify():
+#//     sys.path.append(str(Path(__file__).parent / 'Notify')) # Path needs update if kept
+#//     notify = importlib.import_module('notifyMain')
+#//     if hasattr(notify, 'main'):
+#//         notify.main()
+#//     else:
+#//         raise RuntimeError('Notify/notifyMain.py未找到main函数')
+
+# 监听messages表并联动
+def monitor_and_orchestrate(interval=5):
+    conn = get_db_conn()
+    last_update = get_latest_update_time(conn)
+    print(f"初始messages表更新时间: {last_update}")
+    while True:
+        time.sleep(interval)
+        try:
+            current_update = get_latest_update_time(conn)
+            if current_update != last_update:
+                print(f"检测到messages表有更新: {current_update}, 开始联动执行...")
+                run_filter_llm()
+                run_todogen_llm()
+                last_update = current_update
+            else:
+                print("无更新，继续监听...")
+        except Exception as e:
+            print(f"监听或执行过程中发生错误: {e}")
+
+if __name__ == "__main__":
+    monitor_and_orchestrate(interval=5)

LLM/requirements.txt

@@ -0,0 +1,12 @@
+black
+python-dateutil
+mysql-connector
+mysql-connector-python>=8.0.0
+schedule>=1.2.0
+pyyaml>=6.0.1 
+python-dotenv
+pymysql
+PyYAML
+openai
+tqdm
+

LLM/todogen_LLM/FalsePositive_few_shot.txt

@@ -0,0 +1,31 @@
+# False Positive Few-Shot Examples
+# (Non-actionable messages or pure notifications)
+# 请用实际的、有代表性的例子替换以下内容
+
+## Example 1 (纯粹通知)
+Input Message:
+{"123456789":"[通知]您的账户安全设置已更新。"}
+
+Expected Output JSON:
+```json
+{"123456789":{"is_todo": false, "end_time":"null","location":"null","todo_content":"账号安全更新", "urgency": "unimportant"}}
+```
+
+## Example 2 (垃圾/广告信息)
+Input Message:
+{"987654321":"【优惠促销】限时抢购！全场商品低至一折，点击链接查看详情：xxx.com"}
+
+Expected Output JSON:
+```json
+{"987654321":{"is_todo": false, "end_time":"null","location":"null","todo_content":"广告推销", "urgency": "unimportant"}}
+```
+
+## Example 3 (已完成/过期信息)
+Input Message:
+{"112233445":"[提醒]您昨天预约的会议已结束。"}
+
+Expected Output JSON:
+```json
+{"112233445":{"is_todo": false, "end_time":"null","location":"null","todo_content":"过期内容", "urgency": "unimportant"}}
+``` 
+

LLM/todogen_LLM/TruePositive_few_shot.txt

@@ -0,0 +1,59 @@
+# True Positive Few-Shot Examples
+# (Actionable messages that should result in a to-do)
+
+## Example 1
+Input Message:
+{"323231519":"[2条]李JK老师-1228: 【深势科技】Bohrium用户您好，您有共计余额20.00元的体验卡将于2024-12-24 23:59:59到期，请及时使用。"}
+
+Expected Output JSON:
+```json
+{"323231519":{"is_todo": true, "end_time":"2024-12-24 23:59:59","location":"线上平台:【深势科技】Bohrium","todo_content":"请尽快使用20元体验卡", "urgency": "important"}}
+```
+
+## Example 2
+Input Message:
+{"331150112": "开始日期为2025-03-31T15:01:37，内容源于'ASAP Sample'，[2条]AlisaGG: 【小象超市】您好，我是小象超市骑手，【美团智能外卖柜】您的外卖已送至新一代产业园2栋4号柜（面向马路侧），格口号：27，取件码：9310，存柜超过12小时将被清理，请及时取件。"}
+
+Expected Output JSON:
+```json
+{"331150112": {"is_todo": true, "end_time": "2025-04-01T03:01:37", "location": "线下:新一代产业园2栋4号柜", "todo_content": "取快递(格口号:27, 取件码:9310)", "urgency": "urgent"}}
+```
+
+## Example 3
+Input Message:
+{"331150111": "开始日期为2025-03-31T15:01:36，内容源于'ASAP Sample'，AlisaGG: 【驿收发】您的邮政包裹已到凯丰花园2栋驿站，请23:00前凭5-5-6530来取，详询18320926368"}
+
+Expected Output JSON:
+```json
+{"331150111": {"is_todo": true, "end_time": "2025-03-31T23:00:00", "location": "线下:凯丰花园2栋驿站", "todo_content": "取快递(单号:5-5-6530)", "urgency": "important"}}
+```
+
+## Example 4
+Input Message:
+{"323231510":"[3条]王斯煜[表情]Vince 黑客松nv: 【中通快递】包裹已到深圳光明正大城商业街103号店，取件码3-5-1092。到店扫"取件二维码"，线上查询更方便！询19128399078"}
+
+Expected Output JSON:
+```json
+{"323231510":{"is_todo": true, "end_time":"null","location":"线下地点:深圳光明正大城商业街103号店","todo_content":"快递取件,取件码为3-5-1092", "urgency": "important"}}
+```
+
+## Example 5
+Input Message:
+{"323231172":"[24条]斯煜[表情]Vince: 【高德打车】您有车费尚未支付，为不影响乘车信用，请前往高德地图app处理，或点击l.amap.com/2i0mBifpr 支付"}
+
+Expected Output JSON:
+```json
+{"323231172":{"is_todo": true, "end_time":"null","location":"线上平台: 高德地图app","todo_content":"未支付车费，请点击l.amap.com/2i0mBifpr 支付", "urgency": "urgent"}}
+``` 
+
+## Example 6
+Input Message:
+{"405091409":"【圆通速递】快件尾号1014的包裹已送至（家门口）详询18038103314"}
+
+Expected Output JSON:
+```json
+{"405091409":{"is_todo": true, "end_time":"null","location":"线下:家门口","todo_content":"取圆通快递(尾号1014)", "urgency": "important"}}
+
+
+
+

LLM/todogen_LLM/compare_data.py

@@ -0,0 +1,209 @@
+import json
+import os
+import sys
+import copy
+from config_loader import get_paths
+from datetime import datetime
+
+from export_todolist import export_todolist_to_json
+from receiving_useful_messages import main
+
+# --- 配置与辅助函数 ---
+sys.stdout.reconfigure(encoding='utf-8')
+
+def convert_datetime(obj):
+    """自定义JSON序列化处理器"""
+    if isinstance(obj, datetime):
+        return obj.isoformat()
+    raise TypeError(f"Type {type(obj)} not serializable")
+
+def load_json_data(file_path):
+    """加载JSON文件数据，处理错误并确保返回列表"""
+    if not os.path.exists(file_path):
+        print(f"[错误] 文件未找到: {file_path}")
+        return None
+    try:
+        with open(file_path, 'r', encoding='utf-8') as f:
+            data = json.load(f)
+        if isinstance(data, list):
+            return data
+        else:
+            print(f"[错误] 文件格式不正确，预期为列表: {file_path}")
+            return None
+    except json.JSONDecodeError:
+        print(f"[错误] JSON解码失败: {file_path}")
+        return None
+    except Exception as e:
+        print(f"[错误] 加载文件时发生未知错误 {file_path}: {str(e)}")
+        return None
+
+def generate_unique_id(base_id, existing_ids_set):
+    """生成唯一的 ID (base_id_upd, base_id_upd_1, ...)"""
+    potential_id = f"{str(base_id)}_upd"
+    counter = 1
+    while potential_id in existing_ids_set:
+        potential_id = f"{str(base_id)}_upd_{counter}"
+        counter += 1
+    return potential_id
+
+# --- 核心处理逻辑函数 ---
+def process_record(item_r, existing_message_ids, existing_todo_contents, existing_message_id_to_record, all_known_message_ids, stats):
+    """
+    处理来自 result1.json 的单条记录，根据规则决定操作。
+    返回: 要添加到 compare.json 的记录 (字典) 或 None。
+    同时更新 stats 字典和 all_known_message_ids 集合。
+    """
+    record_to_save = None # 初始化返回值
+
+    try:
+        r_message_id_str = str(item_r['message_id'])
+        r_todo_content_str = str(item_r['todo_content'])
+    except KeyError as e:
+        print(f"[警告] result1.json 中的记录缺少键 {e}，已跳过: {item_r}")
+        stats['skipped_missing_keys'] += 1
+        return None
+    except TypeError as e:
+         print(f"[警告] result1.json 中的记录键值类型错误 {e}，已跳过: {item_r}")
+         stats['skipped_missing_keys'] += 1
+         return None
+
+    if r_message_id_str not in existing_message_ids:
+        # --- 情况 1: 新 message_id ---
+        if r_todo_content_str not in existing_todo_contents:
+            # 1.1: 新 todo_content -> 保存
+            record_to_save = item_r
+            all_known_message_ids.add(r_message_id_str) # 追踪新 ID
+            stats['saved_new_id_new_content'] += 1
+        else:
+            # 1.2: 已存在 todo_content -> 打印
+            print("-" * 30)
+            print(f"打印 (新 message_id: {r_message_id_str}, 但 todo_content 已存在):")
+            print(json.dumps(item_r, indent=2, ensure_ascii=False, default=convert_datetime))
+            print("-" * 30)
+            stats['printed_new_id_existing_content'] += 1
+    else:
+        # --- 情况 2: 已存在 message_id ---
+        record_e = existing_message_id_to_record.get(r_message_id_str) # 获取现有记录
+        if record_e is None: 
+            # 理论上不应发生，因为 ID 在 existing_message_ids 中
+            print(f"[警告] ID {r_message_id_str} 在集合中但在字典中找不到？跳过。")
+            stats['skipped_internal_error'] = stats.get('skipped_internal_error', 0) + 1 # 新增统计
+            return None 
+            
+        e_todo_content_str = str(record_e.get('todo_content', '')) # 安全获取
+
+        if r_todo_content_str != e_todo_content_str:
+            # 2.1: todo_content 不同 -> 修改 ID 并保存
+            new_unique_id = generate_unique_id(r_message_id_str, all_known_message_ids)
+            # all_known_message_ids.add(new_unique_id) # 追踪新生成的 ID
+
+            modified_item_r = copy.deepcopy(item_r)
+            # modified_item_r['message_id'] = new_unique_id
+            record_to_save = modified_item_r
+            stats['saved_modified_id_diff_content'] += 1
+        else:
+            # 2.2: todo_content 相同 -> 打印
+            print("-" * 30)
+            print(f"打印 (message_id: {r_message_id_str} 已存在, todo_content 相同):")
+            print("来自 result1.json:")
+            print(json.dumps(item_r, indent=2, ensure_ascii=False, default=convert_datetime))
+            print("-" * 30)
+            stats['printed_existing_id_same_content'] += 1
+
+    return record_to_save
+
+# --- 主函数 ---
+def compare_and_generate_updates():
+    """主函数：加载数据、处理、保存和打印统计信息"""
+    paths = get_paths()
+    data_dir = paths['data_dir']
+
+    # +++ 新增: 定义 compare_output_file 路径 +++
+    compare_output_file = os.path.join(data_dir, "compare.json")  # 明确输出路径
+
+     # 1. 主动触发数据导出流程，获取导出的 JSON 文件路径
+    extracted_list_path = export_todolist_to_json()  # 返回 todolist_export.json 的路径
+    if not extracted_list_path or not os.path.exists(extracted_list_path):
+        print("[错误] 导出 todolist 数据失败，流程终止。")
+        return
+
+    # 2. 主动触发消息处理流程，生成 result1.json
+    result1_path = main()  # 返回 result1.json 的路径
+    if not result1_path or not os.path.exists(result1_path):
+        print("[错误] 生成 result1.json 失败，流程终止。")
+        return
+
+    # 3. 加载数据
+    result1_data = load_json_data(result1_path)
+    extracted_data = load_json_data(extracted_list_path)
+    if result1_data is None or extracted_data is None:
+        print("[错误] 数据加载失败，流程终止。")
+        return
+
+    # --- 创建查找结构 ---
+    try:
+        existing_message_ids = {str(item['message_id']) for item in extracted_data if 'message_id' in item}
+        existing_message_id_to_record = {str(item['message_id']): item for item in extracted_data if 'message_id' in item}
+        existing_todo_contents = {str(item['todo_content']) for item in extracted_data if 'todo_content' in item}
+    except (KeyError, TypeError) as e:
+        print(f"[错误] extracted_list.json 文件处理失败: {e}。请检查文件内容和格式。")
+        return
+
+    # --- 初始化 ---
+    records_for_compare_json = []
+    all_known_message_ids = set(existing_message_ids)
+    stats = { # 使用字典来存储统计数据
+        'processed': 0,
+        'skipped_missing_keys': 0,
+        'saved_new_id_new_content': 0,
+        'printed_new_id_existing_content': 0,
+        'saved_modified_id_diff_content': 0,
+        'printed_existing_id_same_content': 0,
+        'skipped_internal_error': 0 # 用于 process_record 内部错误
+    }
+
+    print("[信息] 开始比较和处理数据...")
+    # --- 主循环 ---
+    for item_r in result1_data:
+        stats['processed'] += 1
+        record_to_save = process_record(
+            item_r,
+            existing_message_ids,
+            existing_todo_contents,
+            existing_message_id_to_record,
+            all_known_message_ids,
+            stats # 传递 stats 字典用于更新
+        )
+        if record_to_save is not None:
+            records_for_compare_json.append(record_to_save)
+
+    # --- 写入文件 ---
+    try:
+        with open(compare_output_file, 'w', encoding='utf-8') as f:
+            json.dump(records_for_compare_json, f, indent=2, ensure_ascii=False, default=convert_datetime)
+        print(f"✅ 成功生成 compare.json 文件，包含 {len(records_for_compare_json)} 条记录。")
+    except Exception as e:
+        print(f"[错误] 写入 compare.json 文件时发生错误: {str(e)}")
+
+    # --- 打印统计 ---
+    print("=" * 40)
+    print("处理统计:")
+    print(f"  处理 result1.json 记录总数: {stats['processed']}")
+    print(f"  跳过 (缺少关键键或类型错误): {stats['skipped_missing_keys']}")
+    if stats['skipped_internal_error'] > 0:
+       print(f"  跳过 (内部逻辑错误): {stats['skipped_internal_error']}")
+    print("-" * 20)
+    print("  写入 compare.json:")
+    print(f"    - 新 message_id, 新 todo_content: {stats['saved_new_id_new_content']}")
+    print(f"    - 修改后 message_id (因冲突且 todo_content 不同): {stats['saved_modified_id_diff_content']}")
+    print("-" * 20)
+    print("  打印到控制台:")
+    print(f"    - 新 message_id, 但 todo_content 已存在: {stats['printed_new_id_existing_content']}")
+    print(f"    - message_id 已存在, todo_content 相同: {stats['printed_existing_id_same_content']}")
+    print("=" * 40)
+
+
+    return records_for_compare_json
+
+if __name__ == "__main__":
+    compare_and_generate_updates()  # 触发整个流程

LLM/todogen_LLM/config_loader.py

@@ -0,0 +1,34 @@
+# config_loader.py
+import yaml
+from pathlib import Path
+
+def load_config():
+    config_path = Path(__file__).parent / "todogen_LLM_config.yaml"
+    with open(config_path, 'r', encoding='utf-8') as f:
+        return yaml.safe_load(f)
+
+CONFIG = load_config()
+
+def get_mysql_config():
+    return {
+        **CONFIG['mysql'],
+        'ssl_ca': str(Path(__file__).parent / CONFIG['mysql']['ssl_ca'])
+    }
+
+def get_openai_config():
+    return CONFIG['openai']
+
+def get_paths():
+   config = load_config()
+   base = Path(__file__).resolve().parent  # 定位到todogen_LLM目录
+   return {
+        'base_dir': base,
+        'data_dir': base / config['paths']['data_dir'],
+        'logging_dir': base / config['paths']['logging_dir']
+    }
+
+def get_processing_config():
+    return CONFIG['processing']
+
+def get_defaults_config():
+    return CONFIG['defaults']

LLM/todogen_LLM/database_of_messages.py

@@ -0,0 +1,221 @@
+# database_of_messages.py
+from config_loader import get_paths
+import mysql.connector
+from datetime import datetime
+from pathlib import Path
+import sys
+from config_loader import get_mysql_config, get_defaults_config, get_processing_config
+import concurrent.futures  # 必须添加的模块导入
+from concurrent.futures import ThreadPoolExecutor  # 关键修复导入
+from tqdm import tqdm
+import logging # 导入 logging
+
+# 获取 logger 实例
+logger = logging.getLogger(__name__)
+
+sys.stdout.reconfigure(encoding='utf-8')
+
+def process_row(args):
+    """多线程处理单行数据"""
+    columns, row = args
+    row_dict = {}
+    for col_name, value in zip(columns, row):
+        if isinstance(value, datetime):
+            row_dict[col_name] = value.isoformat()
+        elif isinstance(value, int):
+            row_dict[col_name] = str(value)
+        else:
+            row_dict[col_name] = str(value)
+    message_id = str(row_dict.get("message_id", ""))
+    return (message_id, row_dict)
+
+def async_main() -> dict:
+    """带进度条和多线程的版本 - 添加日志记录"""
+
+    paths = get_paths()
+    (paths['data_dir']).mkdir(exist_ok=True) # 确保输出目录存在
+
+    logger.info("开始执行 async_main 获取所有消息...")
+    db_config = get_mysql_config()  # 从配置加载数据库连接信息
+    processing_config = get_processing_config() # 从配置加载处理参数
+    db_fetch_workers = processing_config.get('db_fetch_workers', 4) # 获取数据库读取并发度，默认4
+    
+    conn = None # 初始化 conn
+    cursor = None # 初始化 cursor
+    try:
+        logger.debug("尝试连接数据库 (async_main)...")
+        conn = mysql.connector.connect(
+            user=db_config['user'],
+            password=db_config['password'],
+            host=db_config['host'],
+            port=db_config['port'],
+            database=db_config['database'],
+            ssl_ca=db_config['ssl_ca'],
+            ssl_disabled=False
+        )
+        logger.info("✅ 数据库连接成功 (async_main)")
+        cursor = conn.cursor()
+        logger.info("开始执行查询: SELECT * FROM Messages")
+        cursor.execute("SELECT * FROM Messages")
+        result = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description]
+        logger.info(f"数据库查询完成，获取到 {len(result)} 条原始记录。")
+
+        data = {}
+        # 明确设置 max_workers
+        with ThreadPoolExecutor(max_workers=db_fetch_workers) as executor, \
+             tqdm(total=len(result), desc="数据获取进度") as pbar:
+            
+            future_to_row = {
+                executor.submit(process_row, (columns, row)): row
+                for row in result
+            }
+            
+            processed_count = 0
+            for future in concurrent.futures.as_completed(future_to_row):
+                try:
+                    message_id, row_dict = future.result()
+                    data[message_id] = row_dict
+                    processed_count += 1
+                except Exception as exc:
+                    logger.error(f"处理单行数据时出错: {exc}", exc_info=True)
+                pbar.update(1)
+        logger.info(f"数据行处理完成，成功处理 {processed_count}/{len(result)} 行。")
+        return data
+
+    except mysql.connector.Error as e:
+        # 记录数据库操作错误
+        logger.error(f"❌ 数据库操作错误 (async_main): {e}", exc_info=True)
+        return {}
+    except Exception as e:
+        # 记录其他可能的错误
+        logger.exception(f"❌ async_main 中发生未知错误: {e}") # 使用 exception 自动记录 traceback
+        return {}
+    finally:
+        if cursor:
+            cursor.close()
+        if conn and conn.is_connected():
+            conn.close()
+            logger.info("ℹ️ 数据库连接已关闭 (async_main)。")
+    
+"""
+后面部分即为上传数据部分,切不可搞错
+
+"""
+def upload_to_todolist(data: dict):
+    """将转换后的JSON数据上传到todolist表 (使用 executemany 批量插入) - 添加日志记录"""
+    logger.info(f"开始执行 upload_to_todolist，准备处理 {len(data)} 条输入数据...")
+    db_config = get_mysql_config()  # 直接使用从配置加载的字典
+
+    rows_to_insert = [] # 用于收集待插入的数据行
+    skipped_count = 0
+
+    # 获取默认值配置
+    defaults_config = get_defaults_config()
+    default_todo_status = defaults_config.get('todo_status', 'doing')
+    default_urgency_status = defaults_config.get('urgency_status', 'unimportant')
+
+    logger.debug("开始遍历和转换数据用于批量插入...")
+    for item_key, item in data.items(): # 使用 item_key (可能是 message_id) 记录日志
+        try:
+            # 必填字段处理
+            # 注意：todolist 表结构似乎没有 todo_id 列，插入语句中也没有，这里假设它是 message_id 或其他需要处理的 ID
+            # 假设 user_id 是从 item 中获取���如果不是，需要调整来源
+            # user_id = int(item["user_id"]) 
+            user_id = int(item.get("user_id", 0)) # 示例：如果 item 中没有 user_id，则使用默认值 0，需要确认
+            
+            # 检查 date 字段是否存在且有效
+            date_str = item.get("date")
+            if not date_str:
+                 raise ValueError("缺少 'date' 字段")
+            start_time = datetime.fromisoformat(date_str.replace("T", " "))
+                
+            todo_content = item["todo_content"]
+            
+            # 选填字段处理
+            end_time_str = item.get("end_time")
+            end_time = datetime.fromisoformat(end_time_str) if end_time_str else None
+                
+            location = item.get("location", "")[:255]  # 截断超长内容
+            
+            # 使用从配置加载的默认值
+            todo_status = item.get("todo_statu", default_todo_status)
+            urgency_status = item.get("urgency_statu", default_urgency_status)
+
+            # 将有效数据添加到待插入列表
+            rows_to_insert.append((
+                # 注意：根据 INSERT 语句调整这里的字段顺序和来源
+                user_id,      # user_id
+                start_time,   # start_time
+                end_time,     # end_time
+                location,     # location
+                todo_content, # todo_content
+                todo_status,  # todo_statu
+                urgency_status # urgency_statu
+            ))
+            
+        except (KeyError, ValueError, TypeError) as e: # 捕捉更多可能的错误
+            skipped_count += 1
+            # 替换 print 为 logger.warning
+            logger.warning(f"⚠️ 跳过无效数据 (来自 key {item_key}, 原因: {str(e)}) - 原始数据: {item}")
+            continue
+    
+    logger.debug("数据转换完成。")
+
+    if skipped_count > 0:
+        # 替换 print 为 logger.info
+        logger.info(f"ℹ️ 共跳过 {skipped_count} 条无效数据。")
+
+    if not rows_to_insert:
+        # 替换 print 为 logger.info
+        logger.info("ℹ️ 没有有效数据需要插入到数据库。")
+        return
+
+    logger.info(f"准备将 {len(rows_to_insert)} 条有效数据批量插入数据库...")
+    cnx = None
+    cursor = None
+    try:
+        logger.debug("尝试连接数据库 (upload_to_todolist)...")
+        cnx = mysql.connector.connect(**db_config)
+        logger.info("✅ 数据库连接成功 (upload_to_todolist)")
+        cursor = cnx.cursor()
+
+        # 预处理插入语句 (保持不变)
+        insert_query = """
+        INSERT INTO todolist (
+            user_id, 
+            start_time, 
+            end_time, 
+            location, 
+            todo_content,
+            todo_statu,
+            urgency_statu
+        ) VALUES (%s, %s, %s, %s, %s, %s, %s)
+        """
+
+        logger.info("开始执行批量插入 (executemany)...")
+        cursor.executemany(insert_query, rows_to_insert)
+        rowcount = cursor.rowcount # 获取影响的行数
+        logger.info("批量插入执行完毕，尝试提交事务...")
+        cnx.commit()
+        logger.info(f"✅ 成功批量插入 {rowcount} 条记录到 todolist 表")
+
+    except mysql.connector.Error as err:
+        # 替换 print 为 logger.error
+        logger.error(f"❌ 数据库批量插入错误: {err}", exc_info=True)
+        logger.warning("尝试回滚数据库事务...")
+        try:
+            if cnx and cnx.is_connected():
+                cnx.rollback()
+                logger.warning("数据库事务已回滚。")
+        except Exception as rollback_err:
+            logger.error(f"尝试回滚事务时出错: {rollback_err}", exc_info=True)
+    except Exception as e:
+        logger.exception(f"❌ upload_to_todolist 中发生未知错误: {e}")
+    finally:
+        if cursor:
+            cursor.close()
+        if cnx and cnx.is_connected():
+            cnx.close()
+            # 替换 print 为 logger.info
+            logger.info("ℹ️ 数据库连接已关闭 (upload_to_todolist)。")

LLM/todogen_LLM/export_todolist.py

@@ -0,0 +1,59 @@
+# export_todolist.py
+import json
+from pathlib import Path
+import mysql.connector
+from config_loader import get_mysql_config, get_paths
+from datetime import datetime  # 新增导入
+import sys
+
+sys.stdout.reconfigure(encoding='utf-8')
+
+def convert_datetime(obj):
+    """自定义JSON序列化处理器"""
+    if isinstance(obj, datetime):
+        return obj.isoformat()
+    raise TypeError(f"Type {type(obj)} not serializable")
+
+def export_todolist_to_json():
+    """导出todolist表数据到JSON文件"""
+    try:
+        # 获取配置
+        db_config = get_mysql_config()
+        paths = get_paths()
+        
+        # 建立数据库连接
+        conn = mysql.connector.connect(**db_config)
+        cursor = conn.cursor(dictionary=True)
+        
+        # 执行查询
+        cursor.execute("SELECT * FROM todolist")
+        results = cursor.fetchall()
+        
+        # 创建输出目录
+        output_dir = Path(paths['data_dir'])
+        output_dir.mkdir(parents=True, exist_ok=True)
+        
+        # 保存文件（增加cls参数）
+        output_path = output_dir / "todolist_export.json"
+        with open(output_path, 'w', encoding='utf-8') as f:
+            json.dump(results, f, 
+                     indent=2, 
+                     ensure_ascii=False,
+                     default=convert_datetime)  # 关键修改
+            
+        print(f"✅ 成功导出 {len(results)} 条记录到 {output_path}")
+
+        return str(output_path)
+        
+    except mysql.connector.Error as err:
+        print(f"[错误] 数据库错误: {err}")  # 移除了Unicode符号
+    except Exception as e:
+        print(f"[错误] 发生异常: {str(e)}")
+        return None
+    finally:
+        if 'conn' in locals() and conn.is_connected():
+            cursor.close()
+            conn.close()
+
+if __name__ == "__main__":
+    export_todolist_to_json()

LLM/todogen_LLM/filter_message_list.py

@@ -0,0 +1,49 @@
+import mysql.connector
+from pathlib import Path
+import sys
+from config_loader import get_mysql_config
+
+sys.stdout.reconfigure(encoding='utf-8')
+
+def get_message_ids():
+    current_dir = Path(__file__).parent.absolute()
+    # ssl_ca_path = current_dir / "DigiCertGlobalRootCA.crt.pem"
+    # message_ids = []
+    config = get_mysql_config()
+
+    try:
+        # 建立数据库连接
+        cnx = mysql.connector.connect(
+            user=config['user'],
+            password=config['password'],
+            host=config['host'],
+            port=config['port'],
+            database=config['database'],
+            ssl_ca=config['ssl_ca'],
+            ssl_disabled=False
+        )
+
+        cursor = cnx.cursor()
+        
+        # 执行查询
+        cursor.execute("SELECT message_id FROM filter_message")
+        results = cursor.fetchall()
+        
+        # 提取为纯数字列表
+        message_ids = [row[0] for row in results]
+        
+        cursor.close()
+        cnx.close()
+        print(f"成功获取 {len(message_ids)} 条message_id")
+        
+    except mysql.connector.Error as err:
+        print(f"数据库错误: {err}")
+    except Exception as e:
+        print(f"发生异常: {str(e)}")
+    
+    return message_ids
+
+if __name__ == '__main__':
+    id_list = get_message_ids()
+    print("\n提取结果：")
+    print(id_list)

LLM/todogen_LLM/filter_useful_data_to_dict.py

@@ -0,0 +1,163 @@
+# filter_useful_data_to_dict.py
+from database_of_messages import async_main  # 确保导入的是同步函数
+from typing import List, Dict
+import re
+import json
+import datetime
+from filter_message_list import get_message_ids
+from pathlib import Path # 确保导入 Path
+import logging
+from config_loader import get_paths # 确保导入 get_paths
+
+# --- 在顶层获取 logger 实例 ---
+logger = logging.getLogger(__name__)
+
+# 移除所有异步装饰器和await调用
+def fetch_target_messages(target_ids: List[int]) -> Dict[str, Dict]:
+    """核心函数1：获取指定message_id的原始数据"""
+    logger.info("🛜 正在获取目标消息原始数据...")
+    
+    # 直接调用 async_main，不传递参数
+    all_data = async_main()
+    
+    str_ids = {str(msg_id) for msg_id in target_ids}
+    filtered_data = {k: v for k, v in all_data.items() if k in str_ids}
+    
+    logger.info(f"✅ 找到 {len(filtered_data)}/{len(target_ids)} 条目标消息")
+    return filtered_data
+
+def format_messages(raw_data: Dict[str, Dict]) -> Dict[str, str]:
+    """核心函数2：格式化消息为指定字符串"""
+    logger.info("🔄 正在进行数据格式化...")
+    
+    formatted = {}
+    for msg_id, details in raw_data.items():
+        # 修正：date字段在database_of_messages中已被转换为字符串
+        date = details.get("date", "null")
+        if isinstance(date, str):  # 类型检查改为字符串
+            try:
+                # 转换为datetime对象确保格式有效
+                parsed_date = datetime.datetime.fromisoformat(date)
+                date = parsed_date.strftime("%Y-%m-%dT%H:%M:%S")
+            except ValueError:
+                date = "null"
+        
+        sender = details.get("sender", "null").strip("'‘'")
+        content = details.get("content", "null")
+        
+        # 判断sender是否为纯数字
+        if sender.isdigit():
+            formatted_str = f"开始日期为{date}，{content}"
+        else:
+            # 确保外部双引号，内部单引号
+            formatted_str = f"开始日期为{date}，内容源于'{sender}'，{content}"
+        
+        formatted[msg_id] = formatted_str
+    
+    logger.info("🎉 格式化完成")
+    return formatted
+
+def validate_format(formatted_data: Dict[str, str], target_ids: List[int]) -> bool:
+    for msg_id, content in formatted_data.items():
+        if "内容源于" in content and re.search(r"内容源于'(\d+)'", content):
+            raise ValueError(f"❌ 值 {msg_id} 包含数字来源标识")
+    return True
+
+# --- 恢复 get_formatted_data 的标准保存逻辑 ---
+# 移除 output_override_path 参数
+def get_formatted_data(target_ids: List[int]) -> Dict[str, str]:
+    """
+    供其他模块调用的同步接口 - 保存结果到配置文件指定的目录。
+    """
+
+    paths = get_paths()
+    
+    # 替换原有路径构造逻辑
+    output_file_path = paths['data_dir'] / "filter_data.json"
+    
+    # 确保目录存在
+    paths['data_dir'].mkdir(parents=True, exist_ok=True)
+    
+    logger.info("开始执行 get_formatted_data...")
+    raw_data = fetch_target_messages(target_ids)
+    formatted_data = format_messages(raw_data)
+    try:
+        validate_format(formatted_data, target_ids)
+        logger.info("数据格式验证通过。")
+    except ValueError as ve:
+        logger.error(f"❌ 数据格式验证失败: {ve}")
+        logger.warning("数据格式验证失败，但仍将尝试保存当前格式化数据。")
+
+    # === 恢复：保存到配置文件指定的目录 ===
+    logger.info("准备将格式化数据保存到配置文件指定的目录...")
+    output_filename = "filter_data.json"
+    try:
+        paths_config = get_paths()
+        # 从配置读取 data_dir (值为 "output")
+        output_dir_rel = paths_config.get('data_dir', 'output') # 使用 config 值
+
+        # 获取当前脚本所在的目录
+        script_dir = Path(__file__).resolve().parent
+
+        # 构建正确的输出目录绝对路径 (todogen_LLM/output)
+        output_dir_abs = script_dir / output_dir_rel
+        output_file_path = output_dir_abs / output_filename
+
+        # 确保目标目录存在
+        output_dir_abs.mkdir(parents=True, exist_ok=True)
+
+        with open(output_file_path, 'w', encoding='utf-8') as f:
+            json.dump(formatted_data, f, ensure_ascii=False, indent=2)
+        logger.info(f"✅ 格式化数据已成功保存至: {output_file_path}")
+    except KeyError as e:
+         logger.error(f"❌ config.yaml 中缺少路径配置项 'data_dir': {e}")
+    except Exception as e:
+        logger.error(f"❌ 保存 {output_filename} 时发生错误: {e}", exc_info=True)
+    # ===================================================
+
+    logger.info("get_formatted_data 执行完毕。")
+    return formatted_data
+
+# --- 恢复 main 函数的标准调用 ---
+# 移除 output_override_path 参数
+def main(target_ids: List[int]):
+    """主函数，用于直接运行脚本。"""
+    logger.info("执行 main 函数 (用于直接运行脚本)...")
+    # 调用 get_formatted_data (它现在总是保存到配置指定的目录)
+    formatted_data = get_formatted_data(target_ids)
+    if formatted_data:
+        logger.info("main 函数执行完成，格式化数据已生成并尝试保存。")
+    else:
+        logger.warning("main 函数执行完成，但 get_formatted_data 未返回有效数据或保存失败。")
+
+def print_results(raw_data: Dict[str, Dict], formatted_data: Dict[str, str]):
+    print("\n=== 原始数据 ===")
+    print(json.dumps(raw_data, ensure_ascii=False, indent=2))
+    print("\n=== 格式化数据 ===")
+    print(json.dumps(formatted_data, ensure_ascii=False, indent=2))
+
+if __name__ == '__main__':
+    # === 确保日志也初始化 ===
+    # 注意：这里的 setup_logging() 会配置根 logger，
+    # 我们在顶层获取的 logger = logging.getLogger(__name__) 会继承这个配置
+    try:
+        from logger_config import setup_logging
+        setup_logging()
+    except ImportError:
+        # 如果 logger_config 导入失败，进行基本配置
+        logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
+        # logger = logging.getLogger(__name__) # 这行不再需要在这里定义
+        logger.warning("logger_config 未找到，使用基本日志配置。")
+    except Exception as log_setup_err:
+        # logger = logging.getLogger(__name__) # 这行不再需要在这里定义
+        logger.error(f"日志设置失败: {log_setup_err}", exc_info=True)
+        # 仍然进行基本配置作为后备
+        logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
+    # =============================================
+    
+    # 现在 logger 变量肯定存在了
+    logger.info(f"脚本 {__file__} 作为主程序运行...")
+    
+    TARGET_IDS = get_message_ids()
+    
+    main(TARGET_IDS)

LLM/todogen_LLM/jiaoben.py

@@ -0,0 +1,318 @@
+import json
+from pathlib import Path
+import sys
+from datetime import timedelta
+from dateutil.parser import parse
+import mysql.connector
+from mysql.connector import Error
+from config_loader import get_mysql_config, get_paths
+# 修改导入：导入比较函数，移除旧的合并函数
+from compare_data import compare_and_generate_updates
+import logging # 导入 logging
+from logger_config import setup_logging # 导入 setup_logging
+import traceback # 导入 traceback
+
+# === 日志初始化 ===
+setup_logging() # 在所有代码执行前调用
+logger = logging.getLogger(__name__) # 获取 logger 实例
+# =================
+
+sys.stdout.reconfigure(encoding='utf-8')
+
+def get_db_connection():
+    """建立数据库连接"""
+    logger.debug("尝试建立数据库连接...")
+    try:
+        # current_dir = Path(__file__).parent.absolute()
+        # ssl_ca_path = current_dir / "DigiCertGlobalRootCA.crt.pem"
+
+        db_config = get_mysql_config()
+
+        connection = mysql.connector.connect(
+            user=db_config['user'],
+            password=db_config['password'],
+            host=db_config['host'],
+            port=db_config['port'],
+            database=db_config['database'],
+            ssl_ca=db_config['ssl_ca'],
+            ssl_disabled=False
+        )
+        logger.info("✅ 数据库连接成功")
+        return connection
+    except Error as e:
+        # 使用 logger.error 并包含异常信息
+        logger.error(f"❌ 数据库连接失败: {e}", exc_info=True)
+        return None
+
+def process_end_time(item):
+    """处理end_time字段：如果为null则设置为date加1小时"""
+    # 确保 date 和 end_time 存在
+    date_str = item.get("date")
+    end_time_val = item.get("end_time")
+
+    # 检查 end_time 是否为 None 或 "null"
+    if end_time_val is None or end_time_val == "null":
+        # 检查 date 是否有效
+        if date_str and date_str != "null":
+            try:
+                date_obj = parse(date_str)
+                end_time_obj = date_obj + timedelta(hours=1)
+                item["end_time"] = end_time_obj.isoformat()
+            except (ValueError, TypeError):
+                # 如果 date 解析失败，将 end_time 设为 "null"
+                item["end_time"] = "null"
+        else:
+            # 如果 date 无效，将 end_time 设为 "null"
+            item["end_time"] = "null"
+    # 如果 end_time 已有值，则不做处理
+    return item
+
+
+def insert_to_database(data_list):
+    """将处理后的数据插入到数据库"""
+    connection = get_db_connection()
+    if not connection:
+        # get_db_connection 内部已记录错误
+        return False
+
+    if not data_list:
+        logger.info("ℹ️ 没有数据需要插入数据库。")
+        return True # 没有数据也算成功
+
+    logger.info(f"准备将 {len(data_list)} 条记录插入数据库...")
+    try:
+        cursor = connection.cursor()
+
+        # 准备插入SQL - 在字段列表和 VALUES 中加入 message_id
+        insert_query = """
+        INSERT INTO todolist
+        (message_id, user_id, start_time, todo_content, urgency_statu, end_time, location)
+        VALUES (%s, %s, %s, %s, %s, %s, %s)
+        """
+
+        # 准备数据 - 字段映射关系：
+        # message_id -> message_id  <--- 新增
+        # user_id → user_id
+        # date → start_time
+        # todo_content → todo_content
+        # urgency → urgency_statu
+        # end_time → end_time
+        # location → location
+        records_to_insert = []
+        skipped_count = 0
+        for item in data_list:
+            try:
+                # 处理日期格式，添加更健壮的错误处理
+                start_time = parse(item["date"]).strftime('%Y-%m-%d %H:%M:%S') if item.get("date") and item["date"] != "null" else None
+                end_time = parse(item["end_time"]).strftime('%Y-%m-%d %H:%M:%S') if item.get("end_time") and item["end_time"] != "null" else None
+
+                # 确保关键字段存在且不为空
+                # 获取 message_id，假设它应该是整数，如果不是或者为空则设为 None 或其他默认值
+                message_id_str = item.get("message_id", "")
+                message_id = None
+                if message_id_str:
+                    try:
+                        # 假设 message_id 在数据库中是数字类型
+                        # 如果 message_id 可能是非数字，需要调整这里的转换逻辑
+                        # 或者直接作为字符串插入（如果数据库字段允许）
+                        message_id = int(message_id_str)
+                    except ValueError:
+                         # 替换 print 为 logger.warning
+                         logger.warning(f"⚠️ 跳过记录，message_id '{message_id_str}' 不是有效的整数: {item}")
+                         skipped_count += 1
+                         continue
+
+                user_id = item.get("user_id")
+                todo_content = item.get("todo_content")
+                urgency = item.get("urgency", "unimportant") # 提供默认值
+                location = item.get("location", "") # 提供默认值
+
+                # 增加对 message_id 的检查，如果它在数据库中是必需的
+                if location is None or todo_content is None: # 假设 message_id 也是必需的
+                     # 替换 print 为 logger.warning
+                     logger.warning(f"⚠️ 跳过记录，缺少 location, end_time 或 todo_content: {item}")
+                     skipped_count += 1
+                     continue
+
+                # 在 record 元组中加入 message_id
+                record = (
+                    message_id,     # message_id <--- 新增
+                    user_id,        # user_id
+                    start_time,     # start_time (可能为 None)
+                    todo_content,   # todo_content
+                    urgency,        # urgency_statu
+                    end_time,       # end_time (可能为 None)
+                    location        # location
+                )
+
+                records_to_insert.append(record)
+
+            except (ValueError, TypeError, KeyError) as e:
+                 # 替换 print 为 logger.warning
+                 logger.warning(f"⚠️ 处理记录时出错，已跳过: {item}, 错误: {e}")
+                 skipped_count += 1
+                 continue
+
+
+        if skipped_count > 0:
+            # 替换 print 为 logger.info
+            logger.info(f"ℹ️ 在准备插入数据库时跳过了 {skipped_count} 条记录。")
+
+        if not records_to_insert:
+             # 替换 print 为 logger.info
+             logger.info("ℹ️ 没有有效记录可供插入数据库。")
+             return True # 没有有效数据也算操作完成
+
+        # 执行批量插入
+        cursor.executemany(insert_query, records_to_insert)
+        connection.commit()
+        # 替换 print 为 logger.info
+        logger.info(f"✅ 成功尝试插入 {len(records_to_insert)} 条记录到数据库 (受 INSERT IGNORE 影响，实际插入可能更少)")
+        return True
+
+    except Error as e:
+        # 替换 print 为 logger.error 并包含异常信息
+        logger.error(f"❌ 数据库插入失败: {e}", exc_info=True)
+        if connection.is_connected():
+           connection.rollback() # 如果出错则回滚
+           logger.warning("数据库事务已回滚")
+        return False
+    finally:
+        if connection and connection.is_connected():
+            if cursor: # 确保 cursor 存在再关闭
+                cursor.close()
+            connection.close()
+            # 替换 print 为 logger.info
+            logger.info("ℹ️ 数据库连接已关闭。")
+
+# 重命名函数并修改逻辑
+def process_and_insert_updates() -> bool:
+    """
+    从 compare_data 获取更新数据，处理后插入数据库
+    :return: 处理和插入是否成功
+    """
+    logger.info("🚀 开始执行数据更新与插入流程...")
+    try:
+        # 1. 调用 compare_data 获取需要处理的数据列表
+        logger.info("ℹ️ 开始从 compare_data 获取待处理数据...")
+        data_to_process = compare_and_generate_updates()
+
+        if data_to_process is None:
+             logger.error("❌ 从 compare_data 获取数据失败。")
+             return False
+
+        if not data_to_process:
+             logger.info("ℹ️ compare_data 没有返回需要处理的数据。")
+             # 即使没有数据，也认为流程是成功的，只是没有工作可做
+             # 但仍尝试调用 insert_to_database 以处理空列表情况并关闭连接
+             insert_to_database([])
+             return True
+
+        logger.info(f"ℹ️ 从 compare_data 成功获取 {len(data_to_process)} 条待处理记录。")
+
+        result_list = []
+        for item in data_to_process:
+            # 2. 提取和验证字段 (compare_data 返回的结构已基本符合要求)
+            # 我们主要需要处理 end_time 和格式化时间
+            extracted = {
+                "message_id": item.get("message_id", ""), # 保留 message_id 以便调试或未来使用
+                "date": item.get("date", "null"),
+                "location": item.get("location", ""),
+                "end_time": item.get("end_time"), # 先获取原始值
+                "todo_content": item.get("todo_content", ""),
+                "user_id": item.get("user_id", ""),
+                "urgency": item.get("urgency", "unimportant")
+            }
+
+            # 3. 处理 end_time 字段
+            extracted = process_end_time(extracted)
+
+            # 4. 统一格式化日期字段 (插入数据库时会再次格式化，此步可选，但保持一致性)
+            for time_field in ["date", "end_time"]:
+                 current_val = extracted.get(time_field)
+                 if current_val and current_val != "null":
+                    try:
+                        # 尝试解析以验证格式，并转为 ISO 格式
+                        time_obj = parse(current_val)
+                        extracted[time_field] = time_obj.isoformat()
+                    except (ValueError, TypeError):
+                        # 如果解析失败，标记为 "null"
+                        # 替换 print 为 logger.warning
+                        logger.warning(f"⚠️ 警告：无法解析字段 '{time_field}' 的值 '{current_val}'，将设为 null。记录：{item}")
+                        extracted[time_field] = "null"
+
+
+            result_list.append(extracted)
+
+        # 5. 将数据插入数据库
+        logger.info(f"ℹ️ 准备将处理后的 {len(result_list)} 条记录插入数据库...")
+        if not insert_to_database(result_list):
+            logger.error("❌ 数据插入数据库失败。")
+            return False # 插入失败则整个流程失败
+
+        logger.info("✅ 数据处理和插入流程成功完成。")
+        return True
+
+    except Exception as e:
+        # 替换 print 为 logger.exception (自动包含 traceback)
+        logger.exception(f"❌ 处理和插入过程中发生未预期错误: {str(e)}")
+        # print(traceback.format_exc()) # 不再需要手动打印
+        return False
+
+def compare_data() -> dict:
+    # ... (读取 result1.json 和 filter_data.json 的代码) ...
+    try:
+        # ... (json.load 代码) ...
+        logger.info("✅ 成功加载 result1.json 和 filter_data.json")
+    except Exception as e:
+        logger.error(f"❌ 加载 JSON 文件时出错: {e}", exc_info=True)
+        return [] # 或者 None，取决于 process_and_insert_updates 如何处理
+
+    # --- !! 关键在这里：确保 combined_data 被初始化 !! ---
+    combined_data = {} # <--- 这一行非常重要，它定义了变量并初始化为空字典
+    updates_to_insert = []
+    # --- !! 结束关键点 !! ---
+
+    logger.info("🔍 开始比较数据并生成更新...")
+
+    logger.info("📊 数据比较完成。")
+
+    # === 恢复标准保存 todolist_export.json 逻辑 ===
+    export_filename = "todolist_export.json"
+    logger.info(f"准备导出合并后的数据到配置文件指定的目录...")
+    if combined_data:
+        try:
+            paths_config = get_paths()
+            output_dir_rel = paths_config.get('data_dir', 'output') # 使用 config 值 "output"
+            script_dir = Path(__file__).resolve().parent
+            output_dir_abs = script_dir / output_dir_rel # 构建 todogen_LLM/output
+            export_file_path = output_dir_abs / export_filename
+
+            # +++ 添加诊断日志 +++
+            logger.critical(f"JIAOBEN SAVE EXPORT: Target directory: {output_dir_abs.resolve()}")
+            logger.critical(f"JIAOBEN SAVE EXPORT: Target file: {export_file_path.resolve()}")
+            # +++ 结束添加 +++
+
+            output_dir_abs.mkdir(parents=True, exist_ok=True)
+
+            with open(export_file_path, 'w', encoding='utf-8') as f:
+                json.dump(combined_data, f, ensure_ascii=False, indent=2)
+            logger.info(f"✅ 合并数据已成功导出至: {export_file_path}")
+        except KeyError as e:
+            logger.error(f"❌ config.yaml 中缺少路径配置项 'data_dir': {e}")
+        except Exception as e:
+            logger.error(f"❌ 保存 {export_filename} 时发生错误: {e}", exc_info=True)
+    else:
+        logger.warning(f"没有合并数据可以导出到 {export_filename}。")
+    # === 结束恢复 ===
+
+    logger.info(f"compare_data 函数准备返回 {len(updates_to_insert)} 条待插入记录。")
+    return updates_to_insert
+
+if __name__ == "__main__":
+    logger.info("🚀 脚本入口：开始执行数据更新与插入流程...")
+    # 执行处理和插入流程
+    if process_and_insert_updates():
+        logger.info("🎉 流程执行完毕。")
+    else:
+        logger.error("🔥 处理流程失败，请检查日志中的错误信息。")

LLM/todogen_LLM/logger_config.py

@@ -0,0 +1,69 @@
+# logger_config.py
+import logging
+from config_loader import get_paths
+import os
+import datetime
+from logging.handlers import RotatingFileHandler # Use RotatingFileHandler for potential future size limits
+
+LOG_FOLDER = "todogen_LLM/todogen_logging"
+LOG_LEVEL = logging.INFO # Default level, can be changed (e.g., logging.DEBUG)
+
+def setup_logging():
+    """Configures the logging system."""
+    try:
+        # 
+        paths = get_paths()
+        log_dir = paths['logging_dir']
+
+        # 自动创建日志目录
+        log_dir.mkdir(parents=True, exist_ok=True)
+
+        # 修改日志路径生成逻辑
+        today_str = datetime.date.today().strftime('%Y-%m-%d')
+        log_filename = log_dir / f"{today_str}.log"
+
+        # --- Create Formatter ---
+        # Example format: 2023-10-27 15:30:00,123 - INFO - module_name - Log message
+        log_format = logging.Formatter(
+            '%(asctime)s - %(levelname)s - [%(name)s:%(lineno)d] - %(message)s',
+            datefmt='%Y-%m-%d %H:%M:%S'
+        )
+
+        # --- Get Root Logger ---
+        # Configure the root logger - other modules will inherit this
+        logger = logging.getLogger()
+        logger.setLevel(LOG_LEVEL)
+
+        # --- Avoid Adding Handlers Multiple Times ---
+        # Check if handlers already exist to prevent duplication if setup_logging is called more than once
+        if not logger.handlers:
+            # --- File Handler (Appends to daily log file) ---
+            file_handler = logging.FileHandler(log_filename, mode='a', encoding='utf-8')
+            file_handler.setLevel(LOG_LEVEL)
+            file_handler.setFormatter(log_format)
+            logger.addHandler(file_handler)
+
+            # --- Console Handler (Optional - for seeing logs in the terminal) ---
+            console_handler = logging.StreamHandler()
+            console_handler.setLevel(LOG_LEVEL) # Or set a different level for console, e.g., WARNING
+            console_handler.setFormatter(log_format)
+            logger.addHandler(console_handler)
+
+            logger.info("Logging setup complete. Logging to %s", log_filename)
+        else:
+             # This might happen if setup is called again, which shouldn't normally occur
+             # but this prevents duplicate handlers just in case.
+             pass
+             # logger.debug("Logging already configured.")
+
+
+    except Exception as e:
+        # Fallback basic logging if setup fails
+        logging.basicConfig(level=logging.ERROR)
+        logging.error("Failed to configure logging: %s", e, exc_info=True)
+        # Re-raise the exception if needed, or handle appropriately
+        # raise
+
+# Optional: Call setup immediately when this module is imported?
+# Or better, call it explicitly from the main entry point script.
+# setup_logging() # Avoid calling here, call from main script instead.