README.md

---
title: Advanced Music Analysis & Beat-Matched Lyrics Generator
emoji: 🎵
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.22.0
app_file: app.py
pinned: false
license: mit
short_description: AI music analysis with beat-synchronized lyrics
---

# Advanced Music Analysis & Beat-Matched Lyrics Generator

This comprehensive AI-powered application provides advanced music analysis and generates perfectly synchronized lyrics that match the musical structure, rhythm, and emotional content of your audio files.

## 🎯 Key Features

### 🎼 **Comprehensive Music Analysis**
- **Genre Classification**: Automatic detection using [dima806/music_genres_classification](https://huggingface.co/dima806/music_genres_classification)
- **Tempo & Time Signature Detection**: Advanced multi-method analysis (4/4, 3/4, 6/8)
- **Emotional Analysis**: 8-dimensional emotion detection (happy, sad, excited, calm, etc.)
- **Thematic Analysis**: Identifies musical themes (love, triumph, loss, adventure, etc.)
- **Tonal Analysis**: Key detection, mode analysis (major/minor), harmonic complexity
- **Beat Pattern Analysis**: Precise beat tracking and stress pattern identification

### 🎤 **Beat-Synchronized Lyrics Generation**
- **Rhythm-Matched Lyrics**: Each line perfectly aligns with musical phrases and beat patterns
- **Syllable-to-Beat Mapping**: Precise syllable counting and stress pattern matching
- **Custom Requirements Integration**: Add your own creative directions and themes
- **Genre-Specific Optimization**: Tailored for Pop, Rock, Country, Disco, and Metal
- **Flow Analysis**: Ensures natural sentence flow across multiple lines
- **Quality Metrics**: Detailed beat matching and syllable accuracy analysis

### 🎨 **Personalization Features**
- **Custom Prompt Input**: Specify themes, imagery, perspective, style, or content requirements
- **Intelligent Blending**: Merges your requirements with detected musical characteristics
- **Flexible Creative Control**: From simple themes to complex narrative directions

## 🚀 How It Works

1. **Upload Audio**: Support for various audio formats, or record directly
2. **Add Custom Requirements** (Optional): Specify your creative vision
3. **Advanced Analysis**: Multi-layered analysis of musical characteristics:
   - Rhythm and tempo analysis
   - Time signature detection using autocorrelation, pattern matching, and spectral analysis
   - Emotional profiling using valence-arousal mapping
   - Thematic classification based on musical features
   - Beat pattern extraction and stress analysis
4. **Lyrics Generation**: AI creates lyrics using [Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B) that:
   - Match the detected beat patterns and time signature
   - Incorporate detected emotions and themes
   - Follow your custom creative requirements
   - Maintain proper syllable-to-beat ratios for the genre
5. **Quality Analysis**: Comprehensive beat matching analysis with accuracy metrics

## 🎵 Supported Genres for Lyrics Generation

**Full Support** (Analysis + Beat-Matched Lyrics):
- **Pop**: Optimized syllable patterns and emotional expression
- **Rock**: Energetic phrasing with strong beat emphasis
- **Country**: Narrative flow with authentic storytelling patterns
- **Disco**: Rhythmic momentum with dance-friendly phrasing
- **Metal**: Intense expression with dramatic beat alignment

**Analysis Only**: All other genres receive comprehensive musical analysis without lyrics generation.

## 🛠️ Technical Features

### Advanced Analysis Algorithms
- **Multi-Method Time Signature Detection**: Combines autocorrelation, pattern matching, spectral analysis, note density analysis, and tempo-based estimation
- **Emotion Mapping**: 8-dimensional emotion space with valence-arousal coordinates
- **Beat Strength Analysis**: Onset detection with energy and spectral flux analysis
- **Syllable Stress Matching**: CMU Dictionary integration with rule-based fallback

### AI-Powered Generation
- **4-bit Quantization**: Efficient inference on T4 GPU using BitsAndBytesConfig
- **Specialized Prompting**: Genre-aware prompt engineering for optimal results
- **Quality Enforcement**: Automatic syllable limit enforcement and line count validation
- **Flow Optimization**: Sentence continuation analysis for natural lyrical flow

## 📊 Analysis Outputs

### Musical Analysis
- Tempo (BPM) and time signature with confidence scores
- Primary and secondary emotions with confidence percentages
- Musical themes and their relevance scores
- Key signature and mode detection
- Beat pattern visualization

### Lyrics Quality Metrics
- Syllable-to-beat match accuracy
- Stress pattern alignment scores
- Sentence flow quality assessment
- Genre-appropriate range compliance
- Overall rhythmic accuracy percentage

## 🎯 Custom Requirements Examples

**Themes**: "Write about a journey through mountains", "Focus on urban nightlife"
**Imagery**: "Use ocean metaphors", "Include references to light and shadow"
**Perspective**: "From a child's viewpoint", "Nostalgic memories", "Future aspirations"
**Style**: "Conversational tone", "Include internal rhymes", "Simple everyday language"
**Content**: "Avoid melancholy", "Include words 'freedom' and 'horizon'", "Focus on resilience"

## 🔗 Model Credits

- **Genre Classification**: [dima806/music_genres_classification](https://huggingface.co/dima806/music_genres_classification)
- **Lyrics Generation**: [Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B) with 4-bit quantization
- **Audio Processing**: librosa, scipy, numpy for advanced signal processing
- **Linguistic Analysis**: NLTK CMU Dictionary for syllable counting and stress analysis

## 🎪 Try It Now

Experience the future of AI-powered music analysis and lyrics generation. Upload your music and watch as the system creates perfectly synchronized, emotionally resonant lyrics tailored to your creative vision!