metadata

title: Advanced Music Analysis & Beat-Matched Lyrics Generator
emoji: 🎵
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.22.0
app_file: app.py
pinned: false
license: mit
short_description: AI music analysis with beat-synchronized lyrics

Advanced Music Analysis & Beat-Matched Lyrics Generator

This comprehensive AI-powered application provides advanced music analysis and generates perfectly synchronized lyrics that match the musical structure, rhythm, and emotional content of your audio files.

🎯 Key Features

🎼 Comprehensive Music Analysis

Genre Classification: Automatic detection using dima806/music_genres_classification
Tempo & Time Signature Detection: Advanced multi-method analysis (4/4, 3/4, 6/8)
Emotional Analysis: 8-dimensional emotion detection (happy, sad, excited, calm, etc.)
Thematic Analysis: Identifies musical themes (love, triumph, loss, adventure, etc.)
Tonal Analysis: Key detection, mode analysis (major/minor), harmonic complexity
Beat Pattern Analysis: Precise beat tracking and stress pattern identification

🎤 Beat-Synchronized Lyrics Generation

Rhythm-Matched Lyrics: Each line perfectly aligns with musical phrases and beat patterns
Syllable-to-Beat Mapping: Precise syllable counting and stress pattern matching
Custom Requirements Integration: Add your own creative directions and themes
Genre-Specific Optimization: Tailored for Pop, Rock, Country, Disco, and Metal
Flow Analysis: Ensures natural sentence flow across multiple lines
Quality Metrics: Detailed beat matching and syllable accuracy analysis

🎨 Personalization Features

Custom Prompt Input: Specify themes, imagery, perspective, style, or content requirements
Intelligent Blending: Merges your requirements with detected musical characteristics
Flexible Creative Control: From simple themes to complex narrative directions

🚀 How It Works

Upload Audio: Support for various audio formats, or record directly
Add Custom Requirements (Optional): Specify your creative vision
Advanced Analysis: Multi-layered analysis of musical characteristics:
- Rhythm and tempo analysis
- Time signature detection using autocorrelation, pattern matching, and spectral analysis
- Emotional profiling using valence-arousal mapping
- Thematic classification based on musical features
- Beat pattern extraction and stress analysis
Lyrics Generation: AI creates lyrics using Qwen/QwQ-32B that:
- Match the detected beat patterns and time signature
- Incorporate detected emotions and themes
- Follow your custom creative requirements
- Maintain proper syllable-to-beat ratios for the genre
Quality Analysis: Comprehensive beat matching analysis with accuracy metrics

🎵 Supported Genres for Lyrics Generation

Full Support (Analysis + Beat-Matched Lyrics):

Pop: Optimized syllable patterns and emotional expression
Rock: Energetic phrasing with strong beat emphasis
Country: Narrative flow with authentic storytelling patterns
Disco: Rhythmic momentum with dance-friendly phrasing
Metal: Intense expression with dramatic beat alignment

Analysis Only: All other genres receive comprehensive musical analysis without lyrics generation.

🛠️ Technical Features

Advanced Analysis Algorithms

Multi-Method Time Signature Detection: Combines autocorrelation, pattern matching, spectral analysis, note density analysis, and tempo-based estimation
Emotion Mapping: 8-dimensional emotion space with valence-arousal coordinates
Beat Strength Analysis: Onset detection with energy and spectral flux analysis
Syllable Stress Matching: CMU Dictionary integration with rule-based fallback

AI-Powered Generation

4-bit Quantization: Efficient inference on T4 GPU using BitsAndBytesConfig
Specialized Prompting: Genre-aware prompt engineering for optimal results
Quality Enforcement: Automatic syllable limit enforcement and line count validation
Flow Optimization: Sentence continuation analysis for natural lyrical flow

📊 Analysis Outputs

Musical Analysis

Tempo (BPM) and time signature with confidence scores
Primary and secondary emotions with confidence percentages
Musical themes and their relevance scores
Key signature and mode detection
Beat pattern visualization

Lyrics Quality Metrics

Syllable-to-beat match accuracy
Stress pattern alignment scores
Sentence flow quality assessment
Genre-appropriate range compliance
Overall rhythmic accuracy percentage

🎯 Custom Requirements Examples

Themes: "Write about a journey through mountains", "Focus on urban nightlife" Imagery: "Use ocean metaphors", "Include references to light and shadow" Perspective: "From a child's viewpoint", "Nostalgic memories", "Future aspirations" Style: "Conversational tone", "Include internal rhymes", "Simple everyday language" Content: "Avoid melancholy", "Include words 'freedom' and 'horizon'", "Focus on resilience"

🔗 Model Credits

Genre Classification: dima806/music_genres_classification
Lyrics Generation: Qwen/QwQ-32B with 4-bit quantization
Audio Processing: librosa, scipy, numpy for advanced signal processing
Linguistic Analysis: NLTK CMU Dictionary for syllable counting and stress analysis

🎪 Try It Now

Experience the future of AI-powered music analysis and lyrics generation. Upload your music and watch as the system creates perfectly synchronized, emotionally resonant lyrics tailored to your creative vision!