| --- |
| license: apache-2.0 |
| language: |
| - en |
| - tr |
| tags: |
| - merge |
| - dare-ties |
| - finance |
| - marketing |
| - forecasting |
| - tool-use |
| - mergekit |
| pipeline_tag: text-generation |
| new_version: uaytug/fumea-f-dense-v2 |
| --- |
| |
| # FUMEA-F Dense: Frontier Unified Multi-Expert Agent — Financial (Dense) |
|
|
| FUMEA-F Dense is an 8-billion parameter language model created by merging four domain-specialized models using the DARE+TIES method. It consolidates financial analysis, marketing intelligence, market trend detection, and chain-of-thought financial reasoning capabilities into a single dense transformer — suitable for deployment on consumer hardware without MoE routing overhead. |
|
|
| ## Architecture |
|
|
| | Property | Value | |
| |---|---| |
| | Parameters | ~8B | |
| | Architecture | Decoder-only Transformer (Dense) | |
| | Merge Method | DARE+TIES | |
| | Density | 0.75 | |
| | Normalization | Enabled | |
| | int8 Mask | Enabled | |
| | Context Window | 131,072 tokens | |
| | Positional Encoding | RoPE with YaRN scaling (factor 4.0, base 32,768) | |
| | Precision | bfloat16 | |
| | Vocabulary Size | 151,936 | |
|
|
| ## Merge Configuration |
|
|
| Four 8B-parameter models were merged with the following weight distribution, biased toward financial domains: |
|
|
| | Expert Domain | Weight | Role | |
| |---|---|---| |
| | Marketing Intelligence | 0.30 (base) | Brand strategy, campaign analysis, market positioning | |
| | Financial Forecasting | 0.25 | Time-series prediction, technical indicators, quantitative modeling | |
| | Market Trends | 0.20 | E-commerce analytics, consumer behavior, competitive pricing | |
| | Financial Reasoning | 0.25 | Multi-step chain-of-thought, valuation logic, regulatory analysis | |
|
|
| The marketing model served as the base for the merge due to its broad domain coverage, providing the structural foundation onto which specialized financial and trend capabilities were integrated. |
|
|
| ## Capabilities |
|
|
| **Financial Analysis** |
| - Fundamental metrics: EBITDA margin, P/E ratio, PEG ratio, DCF frameworks |
| - Technical analysis: candlestick interpretation, support/resistance identification |
| - Risk assessment: portfolio theory, diversification strategies, stress testing concepts |
|
|
| **Marketing Intelligence** |
| - Campaign performance analysis and ROI attribution |
| - Competitive landscape and positioning assessment |
| - Market sizing, segmentation, and go-to-market strategy |
|
|
| **Market Trend Detection** |
| - E-commerce pricing dynamics and conversion analysis |
| - Consumer behavior pattern recognition |
| - Emerging market opportunity identification |
|
|
| **Financial Reasoning** |
| - Step-by-step derivation of financial conclusions |
| - Multi-factor valuation with explicit reasoning chains |
| - Regulatory compliance analysis with structured argumentation |
|
|
| **Tool Use** |
| Supports structured tool calling via the chat template, including: |
|
|
| - `analyze_ohlcv` — OHLCV time-series analysis with configurable indicators |
| - `web_search` — External information retrieval |
| - `code_executor` — Python code execution |
| - `file_reader` — Structured file parsing |
|
|
| **Extended Context** |
| 128K token context window for processing lengthy financial documents, earnings transcripts, and large datasets in a single pass. |
|
|
| ## Usage |
|
|
| ### Basic Inference |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| "uaytug/fumea-f-dense", |
| torch_dtype=torch.bfloat16, |
| device_map="auto", |
| trust_remote_code=True, |
| ) |
| tokenizer = AutoTokenizer.from_pretrained("uaytug/fumea-f-dense", trust_remote_code=True) |
| |
| messages = [ |
| {"role": "system", "content": "You are a financial analysis assistant."}, |
| {"role": "user", "content": "Explain the PEG ratio and when it is most useful for stock valuation."} |
| ] |
| |
| text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer(text, return_tensors="pt").to(model.device) |
| |
| output = model.generate( |
| **inputs, |
| max_new_tokens=1024, |
| do_sample=True, |
| temperature=0.6, |
| top_p=0.9, |
| repetition_penalty=1.1, |
| ) |
| print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ### Tool Use |
|
|
| ```python |
| tools = [ |
| { |
| "name": "analyze_ohlcv", |
| "description": "Analyze OHLCV time-series data for pattern recognition and trend detection", |
| "parameters": { |
| "type": "object", |
| "properties": { |
| "symbol": {"type": "string", "description": "Trading symbol"}, |
| "ohlcv_data": { |
| "type": "array", |
| "items": { |
| "type": "object", |
| "properties": { |
| "timestamp": {"type": "string"}, |
| "open": {"type": "number"}, |
| "high": {"type": "number"}, |
| "low": {"type": "number"}, |
| "close": {"type": "number"}, |
| "volume": {"type": "number"} |
| } |
| } |
| }, |
| "indicators": { |
| "type": "array", |
| "items": {"type": "string"}, |
| "description": "Technical indicators: RSI, MACD, BB, EMA_20, SMA_50" |
| }, |
| "prediction_horizon": {"type": "integer"} |
| }, |
| "required": ["symbol", "ohlcv_data"] |
| } |
| } |
| ] |
| |
| messages = [ |
| {"role": "user", "content": "Run a technical analysis on TSLA with RSI and Bollinger Bands."} |
| ] |
| |
| text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True) |
| ``` |
|
|
| ## Quantized Versions |
|
|
| GGUF quantizations from F32 to IQ2_M are available at [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) for use with Ollama, LM Studio, llama.cpp, and other GGUF-compatible runtimes. |
| |
| | Quantization | Recommended Use | |
| |---|---| |
| | Q8_0 | Maximum quality, high-end hardware | |
| | Q5_K_M | Strong quality with moderate resource requirements | |
| | Q4_K_M | Best balance of quality and efficiency (recommended) | |
| | Q3_K_M | Reduced quality, constrained environments | |
| | IQ2_M | Experimental, extreme compression | |
| |
| ## Hardware Requirements |
| |
| | Configuration | Minimum VRAM / RAM | |
| |---|---| |
| | Full precision (bfloat16) | 16 GB VRAM | |
| | Q8_0 GGUF | 8.71 GB VRAM | |
| | Q4_K_M GGUF | 5.03 GB VRAM | |
| | Q3_K_S GGUF | 3.77 GB VRAM | |
| | Q2_K GGUF | 3.28 GB VRAM | |
| |
| ## Dense vs. MoE |
| |
| This is the dense variant of the FUMEA-F family. Compared to the MoE version: |
| |
| | Property | FUMEA-F Dense | FUMEA-F MoE | |
| |---|---|---| |
| | Parameters | ~8B | ~24B (~16B active) | |
| | Experts | 1 (unified) | 4 (top-2 routing) | |
| | Min VRAM (bf16) | 16 GB | 48 GB | |
| | Min VRAM (Q4_K_M) | 6 GB | 8 GB | |
| | Inference Speed | Faster (no routing) | Slower (expert selection overhead) | |
| | Specialization | Blended across all domains | Dynamic routing to domain experts | |
| |
| Choose Dense for resource-constrained deployments or when uniform cross-domain performance is preferred. Choose MoE for maximum capability when hardware allows. |
| |
| ## Generation Defaults |
| |
| | Parameter | Value | |
| |---|---| |
| | temperature | 0.6 | |
| | top_p | 0.9 | |
| | repetition_penalty | 1.1 | |
| | max_new_tokens | 8192 | |
| |
| ## Build Process |
| |
| 1. Four 8B-parameter domain-specialized models were merged using DARE+TIES with density 0.75 and normalization enabled |
| 2. Weight distribution was biased toward financial domains (marketing 0.30, forecasting 0.25, trends 0.20, reasoning 0.25) |
| 3. Context window was extended to 128K via YaRN RoPE scaling |
| 4. Tool-use chat template was injected for structured function calling |
| 5. GGUF quantization ladder was generated using llama.cpp with CUDA acceleration |
| |
| ## Related Models |
| |
| - [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) — Quantized GGUF versions of this model |
| - [uaytug/fumea-f](https://huggingface.co/uaytug/fumea-f) — 24B MoE version (4 experts, top-2 routing) |
| - [uaytug/fumea-f-gguf](https://huggingface.co/uaytug/fumea-f-gguf) — Quantized GGUF versions of the MoE model |
| |
| ## Limitations |
| |
| - This is a merged model, not fine-tuned on curated financial datasets. Output quality reflects the combined capabilities of the source models. |
| - Financial predictions and analysis should not be used as sole basis for investment decisions. |
| - Tool-use capability depends on the inference framework supporting structured function calling. |
| - Extended context performance has not been systematically benchmarked beyond the base model's tested range. |
| - As a dense merge, domain-specific performance may be diluted compared to the individual specialist models or the MoE variant. |
| |
| ## License |
| |
| Apache 2.0 |
| |
| ## Citation |
| |
| ```bibtex |
| @misc{FUMEA-F-Dense, |
| author = {Umut Aytuğ Semerci}, |
| title = {FUMEA-F Dense: Dense model of Financial Frontier Unified Multi-Expert Agent}, |
| year = {2026}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/uaytug/fumea-f-dense} |
| } |
| ``` |
