import torch
import torch.nn.functional as F
from torchvision import transforms
import numpy as np

# Core function for temporally consistent DeepDream

def apply_temporal_consistency(current_frame, previous_frame, dream_strength=0.6):
    """
    Apply optical flow and occlusion masking to maintain consistency
    between frames in DeepDream video processing.
    """
    
    # Convert frames to tensors
    current_tensor = transforms.ToTensor()(current_frame)
    prev_tensor = transforms.ToTensor()(previous_frame)
    
    # Calculate optical flow between frames
    flow = calculate_optical_flow(prev_tensor, current_tensor)
    
    # Warp previous frame's features to current frame
    warped_features = F.grid_sample(prev_tensor.unsqueeze(0), 
                                   flow.unsqueeze(0), 
                                   align_corners=True)
    
    # Calculate occlusion mask (where flow fails)
    occlusion_mask = compute_occlusion_mask(prev_tensor, current_tensor, flow)
    
    # Blend current frame with warped previous features
    # This maintains temporal consistency while allowing new features
    blended = (1 - dream_strength) * current_tensor + \
              dream_strength * warped_features.squeeze()
    
    # Apply occlusion masking
    result = torch.where(occlusion_mask > 0.5, 
                        current_tensor, 
                        blended)
    
    return transforms.ToPILImage()(result)

# Helper functions (simplified for clarity)
def calculate_optical_flow(frame1, frame2):
    """Calculate optical flow between two frames"""
    # Implementation using RAFT or FlowNet2
    pass

def compute_occlusion_mask(frame1, frame2, flow):
    """Detect occluded regions where flow estimation fails"""
    # Compare forward and backward flow consistency
    pass

The Mesmerizing Problem That Plagued AI Art

import cv2
import numpy as np
from deepdream import deepdream
from optical_flow import compute_flow

# Core function for temporally consistent DeepDream video

def process_video_frame_temporally(frame, prev_frame, prev_dreamed):
    """
    Apply DeepDream with temporal consistency using optical flow
    to eliminate flickering between video frames.
    """
    
    # 1. Compute optical flow between current and previous frame
    flow = compute_flow(prev_frame, frame)
    
    # 2. Warp previous DeepDream result using computed flow
    warped_prev = cv2.remap(prev_dreamed, flow, None, cv2.INTER_LINEAR)
    
    # 3. Create occlusion mask (where flow estimation is unreliable)
    h, w = frame.shape[:2]
    occlusion_mask = create_occlusion_mask(flow, threshold=0.5)
    
    # 4. Blend current DeepDream with warped previous result
    current_dream = deepdream(frame)
    
    # Use occlusion mask to blend: where reliable, use warped previous;
    # where occluded, use current DeepDream
    blended = np.where(occlusion_mask[..., None], 
                      warped_prev, 
                      current_dream)
    
    # 5. Apply temporal smoothing
    alpha = 0.7  # Weight for temporal consistency
    final_frame = alpha * blended + (1 - alpha) * current_dream
    
    return final_frame.astype(np.uint8)

# Helper function for occlusion detection
def create_occlusion_mask(flow, threshold=0.5):
    """
    Create mask where optical flow is unreliable (occlusions).
    Returns boolean mask where True indicates reliable areas.
    """
    flow_magnitude = np.sqrt(flow[..., 0]**2 + flow[..., 1]**2)
    return flow_magnitude < threshold

Since Google researchers first unveiled DeepDream in 2015, the world has been captivated by its psychedelic, AI-generated hallucinations—eyes peering from clouds, dog faces emerging in architecture, and intricate patterns blooming from ordinary images. The technique, which amplifies patterns that neural networks "see" in images, created an entirely new genre of digital art. But there was always one glaring limitation: when applied to video, DeepDream produced a chaotic, flickering mess that was visually jarring and artistically unusable.

"The fundamental issue was temporal inconsistency," explains AI researcher Dr. Elena Vasquez, who has studied neural network visualization techniques. "Each frame was processed independently, so the hallucinations would jump around randomly between frames. What looked like a beautiful pattern in one frame might completely disappear in the next, only to reappear somewhere else. It was like watching a slideshow of unrelated images rather than a coherent video."

How Optical Flow and Occlusion Masking Fix the Flicker

The breakthrough comes from developer Nemanja Jeremić, who forked an existing PyTorch DeepDream implementation and added sophisticated video processing capabilities. The core innovation lies in two complementary techniques: optical flow warping and occlusion masking.

Optical flow warping analyzes the motion between consecutive video frames, tracking how pixels move from one frame to the next. Instead of starting each frame's DeepDream process from scratch, the algorithm warps the previous frame's hallucinations according to this calculated motion. If a hallucinated pattern moves with an object in the video, it maintains its position relative to that object across frames.

Occlusion masking addresses what happens when objects move in front of or behind each other. "Without occlusion handling, you'd get ghosting effects—hallucinations from background objects would bleed into foreground objects as they moved," Jeremić explains in the project documentation. The system detects these occlusion events and prevents hallucination transfer where it shouldn't occur.

The Technical Architecture

The implementation builds on PyTorch and supports multiple pretrained classifiers including GoogLeNet, VGG, and ResNet architectures. Users can control numerous parameters: the strength of the temporal consistency, which neural network layers to activate, the scale of patterns, and the iteration count per frame. This flexibility allows creators to dial in anything from subtle, dreamlike enhancements to full-blown psychedelic transformations.

"What's clever about this approach is that it doesn't require retraining the neural network or modifying the DeepDream algorithm itself," notes Vasquez. "It's a post-processing framework that intelligently connects the independent frame results. This makes it highly compatible with existing DeepDream workflows while solving the fundamental video problem."

Why This Matters Beyond Trippy Videos

While the immediate application creates stunning visual art, the implications extend further. Temporal consistency is a fundamental challenge in many AI video processing tasks—from style transfer and colorization to super-resolution and frame interpolation.

"The techniques demonstrated here could transfer to other domains," says Jeremić. "Any application where you want to apply an image-based AI transformation to video while maintaining coherence across frames faces similar challenges. Our occlusion masking approach, in particular, could help with video inpainting or object removal tasks."

The open source nature of the project accelerates this potential. With the code publicly available on GitHub, researchers and developers can build upon these techniques rather than starting from scratch. Early adopters have already begun experimenting with the tool, producing everything from dreamlike music videos to enhanced nature documentaries.

Practical Applications Emerging

Film and video artists now have a tool that was previously unavailable. "Before this, creating consistent DeepDream videos required painstaking manual frame-by-frame adjustments or proprietary software," says digital artist Marco Chen. "This democratizes the technique. I can imagine music videos, title sequences, or experimental films that maintain the DeepDream aesthetic without the visual nausea of flickering."

Educational applications also emerge. Neuroscience and AI instructors can now demonstrate how neural networks "see" moving images consistently, showing how patterns propagate through video rather than appearing as disconnected stills. This could help students better understand feature visualization in dynamic contexts.

The Future of AI-Generated Video Art

Jeremić's implementation represents a significant step toward making AI video tools more practical and artistically viable. As the project evolves, several directions seem promising:

• Real-time processing: Currently, the method requires significant computation time. Optimization could lead to near-real-time applications for live video or interactive installations.
• Integration with other models: The temporal consistency framework could be adapted to work with newer architectures like Vision Transformers or diffusion models.
• Parameter learning: Rather than manually setting consistency parameters, future versions might automatically learn optimal settings for different video types.
• Community contributions: As an open source project, additional features and improvements will likely emerge from the community.

"This isn't just about fixing a technical problem," concludes Vasquez. "It's about enabling a new form of expression. When tools become predictable and controllable, artists can work with intention rather than fighting randomness. That's when true creativity flourishes."

The project serves as a reminder that sometimes the most valuable innovations aren't entirely new algorithms, but clever integrations that solve practical problems. By combining established computer vision techniques (optical flow) with neural network visualization (DeepDream), Jeremić has created something greater than the sum of its parts—a tool that finally delivers on the promise of AI-generated video art without the distracting flicker that has limited it for nearly a decade.