|
|
--- |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- marketing |
|
|
- reasoning |
|
|
base_model: |
|
|
- Qwen/Qwen3-8B |
|
|
license: mit |
|
|
library_name: transformers |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# Markethinking |
|
|
## QWEN3-Marketing: Reasoning-LLM for Marketing |
|
|
|
|
|
Markethinking is a domain-specific large language model, adapted from Qwen/Qwen3-8B through finetuning on over 10 billion tokens of curated marketing data. |
|
|
It is the first in our line of models to inherit and preserve reasoning capabilities for domain-specific applications. |
|
|
This **early checkpoint** is released for research, experimentation, and continued development by the community. |
|
|
|
|
|
 |
|
|
|
|
|
**Qwen-Marketing** is a reasoning-optimized language model fine-tuned for marketing tasks. |
|
|
|
|
|
Unlike general-purpose LLMs, Qwen-Marketing specializes in understanding marketing contexts, strategies, and tone. It was trained on proprietary data combined with curated open datasets to ensure performance across real-world business scenarios. |
|
|
|
|
|
Fine-tuned from Qwen1.5 (or relevant base model) |
|
|
Trained on real marketing tasks, prompts, and responses |
|
|
|
|
|
|
|
|
## Use Case & Applications |
|
|
Qwen-Marketing is designed for marketers, brand strategists, product managers, and marketing analysts. |
|
|
|
|
|
Example use cases: |
|
|
- Writing product descriptions in brand voice |
|
|
- Generating campaign ideas or messaging variants |
|
|
- Summarizing customer feedback |
|
|
- Answering marketing-related questions with context-specific reasoning |
|
|
|
|
|
|
|
|
## Model Description |
|
|
|
|
|
*Markethinking* blends the powerful general capabilities of Qwen3-8B with deep domain knowledge from the marketing world. It supports: |
|
|
|
|
|
- Reasoning-driven content generation |
|
|
|
|
|
- Domain-specific language modeling in marketing |
|
|
|
|
|
- Long-context handling (up to 32,768 tokens natively) |
|
|
|
|
|
This checkpoint is instruction-tuned and should be used for research purposes. Use in high-stakes or production settings is not advised. |
|
|
|
|
|
|
|
|
## Model Details |
|
|
|
|
|
| Developed by | [Marketeam](https://www.marketeam.ai/) | |
|
|
|---------------|------------| |
|
|
| Base Model | Qwen/Qwen3-8B | |
|
|
| Architecture | Decoder-only transformer | |
|
|
| Parameters | 8B | |
|
|
| Context Length | 32,768 tokens | |
|
|
| Reasoning | Yes | |
|
|
| Input | Text-only | |
|
|
| Output | Text-only | |
|
|
| Language | English | |
|
|
| Knowledge Cutoff | December 2024 | |
|
|
| License | Apache 2.0 | |
|
|
|
|
|
|
|
|
|
|
|
## Intended Use |
|
|
|
|
|
`Markethinking` is intended for: |
|
|
|
|
|
- Domain-specific Q&A in marketing contexts |
|
|
|
|
|
- Strategic idea development (customer personas, campaign planning) |
|
|
|
|
|
- Marketing content generation (product copy, email sequences, landing pages, ...) |
|
|
|
|
|
⚠️ This early checkpoint isn't aligned for production. Use in controlled environments only. |
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
`Markethinking` was adapted from Qwen3-8B through **marketing-specific reasoning tasks**. |
|
|
|
|
|
We used syntetic data and real-world data. Grounding the model around information and tasks around: |
|
|
|
|
|
- Ad campaigns |
|
|
|
|
|
- Email campaigns |
|
|
|
|
|
- Meetings, Podcasts |
|
|
|
|
|
- Landing pages, Newsletters |
|
|
|
|
|
- Blogs, Books, Websites, Articles |
|
|
|
|
|
- Social Media Posts, Press Releases, Trends |
|
|
|
|
|
- ... |
|
|
|
|
|
~5% general corpus was retained to avoid catastrophic forgetting. |
|
|
|
|
|
Optimization techniques: |
|
|
- Fine-tuning via supervised instruction-following |
|
|
- Prompt-based format with marketing-specific structure |
|
|
- Negative prompt formats to teach safety and relevance |
|
|
|
|
|
## Training |
|
|
|
|
|
We used AWS SageMaker (`p4de.24xlarge`), with 4× NVIDIA A100 (80GB) GPUs. |
|
|
|
|
|
|
|
|
| Param | Value | |
|
|
|---------------|------------| |
|
|
| Optimizer | adamw_torch_fused | |
|
|
| Learning Rate | 4e-4 | |
|
|
| Precision | bf16 | |
|
|
| Gradient Accumulation | 64 steps | |
|
|
| Epochs | 3 | |
|
|
| Max Seq Length | 2500 | |
|
|
| Scheduler | Cosine | |
|
|
| Gradient Checkpointing | Enabled | |
|
|
| FSDP | Full Shard + Transformer Layer Auto-Wrap | |
|
|
| QLoRA | Enabled | |
|
|
|
|
|
|
|
|
|
|
|
## How to use |
|
|
|
|
|
### Use a pipeline as a high-level helper |
|
|
|
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
pipe = pipeline("text-generation", model="marketeam/Qwen-Marketing") |
|
|
messages = [ |
|
|
{"role": "user", "content": "Who are you?"}, |
|
|
] |
|
|
pipe(messages) |
|
|
``` |
|
|
|
|
|
### Load model directly |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("marketeam/Qwen-Marketing") |
|
|
model = AutoModelForCausalLM.from_pretrained("marketeam/Qwen-Marketing") |
|
|
``` |
|
|
|
|
|
The model was trained on proprietary marketing data, as well as open datasets curated by our team: |
|
|
- [`offtopic`](https://huggingface.co/datasets/marketeam/offtopic) – for irrelevant content filtering |
|
|
- [`about`](https://huggingface.co/datasets/marketeam/about) – for tone and brand narrative modeling |
|
|
- [`marketing_user_prompts`](https://huggingface.co/datasets/marketeam/marketing_user_prompts) – for supervised prompt training |
|
|
|
|
|
## Safety & Bias |
|
|
To reduce hallucinations and improve safety: |
|
|
- Negative prompts were included during training (showing the model what *not* to do) |
|
|
- Fine-tuning was applied on real-world, domain-specific data to ensure appropriate outputs in context |
|
|
|
|
|
## Performance & Benchmarking |
|
|
While there is no formal academic benchmark yet, our internal tests ("Marketeam Benchmarketing") show: |
|
|
|
|
|
- Higher relevance and brand tone accuracy |
|
|
- Lower hallucination rate on product-focused queries |
|
|
- Better performance than GPT-4o & DeepSeek & Qwen & LLama on marketing prompts |
|
|
|
|
|
## Deployment & Integration |
|
|
Qwen-Marketing is available as a Hugging Face model and can be deployed: |
|
|
- Via API. |
|
|
- In your own marketing GenAI pipelines. |
|
|
- Embedded in CRM, analytics, or content tools. |
|
|
|
|
|
Clone it, run it locally, or use inference widgets to test. |
|
|
|
|
|
### Prompt Example |
|
|
|
|
|
**User Prompt:** |
|
|
```Write a product launch email for a new AI-based skincare analyzer. Keep it confident, science-driven, and friendly.``` |
|
|
|
|
|
**Qwen-Marketing Output:** |
|
|
``` |
|
|
Subject: Meet Your Skin’s New Best Friend 🧪✨ |
|
|
Body: Discover personalized skincare backed by real science. Our AI Skin Analyzer scans your skin in seconds and gives you the exact ingredients it craves. Say goodbye to guesswork — and hello to glowing confidence. |
|
|
``` |
|
|
|
|
|
## License & Attribution |
|
|
|
|
|
- License: Apache 2.0 |
|
|
- Base model: Qwen/Qwen3-8B |
|
|
|
|
|
Citation |
|
|
@misc{qwen3technicalreport, |
|
|
title={Qwen3 Technical Report}, |
|
|
author={Qwen Team}, |
|
|
year={2025}, |
|
|
eprint={2505.09388}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CL}, |
|
|
url={https://arxiv.org/abs/2505.09388}, |
|
|
} |
|
|
|
|
|
## Contributers |
|
|
|
|
|
[Sahar Millis](https://www.linkedin.com/in/sahar-millis/) [Coby Benveniste](https://www.linkedin.com/in/coby-benveniste/) |
