NebulaeWis/Qwen3-0.6B-Prompt-Gen-alpha-300k

这是一个基于 Qwen/Qwen3-0.6B-Base 进行指令微调的语言模型。

eval loss = 1.02

这是模型的第一个 Alpha 测试版本，提供以下三种功能：

1. 自然语言转标签 (NL to Tag)
2. 标签转自然语言 (Tag to NL)
3. 标签补全 (Tag to Tag)

模型详情

• 基础模型: Qwen/Qwen3-0.6B-Base
• 微调方法: 指令微调 (Instruction Fine-tuning)
• 训练数据: 模型使用了约 30 万条数据进行训练。数据集包含三个指令任务，每个任务约有 9.2 万条训练样本。

如何使用

请使用特定的指令 Token 来引导模型执行相应任务。输入和输出需要使用指定的 XML 格式包裹。

1. 自然语言描述 → Danbooru 标签 (<NLTOTAG>)

• 指令: <NLTOTAG>
• 输入:

  <caption>This illustration features a young girl standing with a smile, looking directly at the viewer. She's wearing a white beret, and her long, light-colored hair is styled with pigtails, adorned with ribbons. She sports round glasses and a white collared shirt with long sleeves, layered with a light orange sweater vest. A pleated skirt that matches the ribbons, and thigh-highs complete her outfit. Her hands are clasped over her chest, possibly holding a book or other object. The background is plain white, emphasizing the character.</caption>
  

• 输出:

  <tags><special>1girl</special><artists></artists><characters></characters><copyrights>original</copyrights><general>skirt, thighhighs, pleated_skirt, smile, white_background, hands_on_own_chest, looking_at_viewer, long_hair, glasses, solo, simple_background, long_sleeves, sketch</general><rating>safe</rating></tags>
  

2. Danbooru 标签 → 自然语言描述 (<TAGTONL>)

• 指令: <TAGTONL>
• 输入:

  <tags><special>2girls</special><artists></artists><characters></characters><copyrights></copyrights><general>bracelet, casual, purple_hair, oekaki, outdoors, white_flower, blouse, jewelry, pendant, drill_hair, dandelion, closed_eyes, smile, short_sleeves, frills</general><rating>safe</rating></tags>
  

• 输出:

  <caption>The image is a vibrant, anime-style illustration of two young women, likely teens, lying together on a bed of green grass dotted with dandelions. The composition features a close-up, overhead perspective. The first girl has brown drill hair styled in two high twintails, secured with pink bows featuring black polka dots, and she's dressed in a polka-dotted dress with frilly details and short sleeves, with a pink heart pendant necklace. She's smiling widely, eyes closed, showing a blush on her cheeks, and is looking at the other girl. The second girl, also wearing a similar style dress with a pink bow, and has pink hair, looks back at her with a smile, hugging her. Both girls have bracelets. The overall atmosphere is bright and cheerful, suggesting a moment of friendship and intimacy in a sunny outdoor setting.</caption>
  

3. 标签补全与优化 (<TAGTOTAG>)

• 指令: <TAGTOTAG>
• 输入:

  <tags><special>2girls</special><artists></artists><characters></characters><copyrights></copyrights><general>looking_at_another, hug, camisole, on_back, blush, oekaki, field, jewelry, dandelion_clock, on_grass, pendant, blouse, short_sleeves, dandelion, frills, bow, casual, smile, sleeveless, outdoors, brown_hair, pink_bow, hair_ribbon, polka_dot, shirt, short_hair, yellow_flower, lying, flower, closed_eyes, bracelet, drill_hair, sparkle, grass, on_side, purple_hair, ribbon, on_ground, white_flower</general><rating>safe</rating></tags>
  

• 输出:

  <tags><special>2girls</special><artists></artists><characters></characters><copyrights></copyrights><general>closed_eyes, hair_ribbon, oekaki, sleeveless, sparkle, hug, pink_bow, white_flower, short_hair, looking_at_another, dandelion_clock, ribbon, pendant, flower, lying, purple_hair, bracelet, smile, bow, brown_hair, frills, blush, jewelry, short_sleeves, on_grass, casual, grass, outdoors, shirt, blouse, field, yellow_flower, camisole, on_back, twintails, polka_dot, on_ground, on_side, dandelion</general><rating>safe</rating></tags>
  

已知问题

1. <TAGTOTAG> 的数据集构建。 可能把输入（需要补全的 tags）和输出（补全后的 tags）都随机 drop 掉了一部分。理论上只应该 drop 输入，这导致模型这边奇怪。
2. 短样本筛选不力。 没有把特别短的 tag 样本过滤。
3. 模型还是太小了！ 0.6b的模型而且dim只开了32

未来计划

• 重新处理数据集： 换成10% dan和full dan进行训练。
• 模型参数调优： dim拉高/换1.5b的qwen

引用

没有引用，孩子不懂事炼着玩的