A NOVEL DUAL-FRAMEWORK FOR AI SYNTHETIC MEDIA DETECTION BASED ON PHYSIOLOGICAL AND LINGUISTIC INCONSISTENCIES

Abstract

The rapid advancement of generative artificial intelligence has intensified the challenge of detecting AI synthesized facial media, commonly known as deepfakes. This study introduces a novel dual-framework that fuses physiological and linguistic inconsistency analysis for robust synthetic media detection. The first component, Spatiotemporal Drift Entropy Mapping (SDEM), quantifies micro-temporal irregularities in facial motion using entropy and spectral variance of 468 FaceMesh landmarks. The second component, Inverse Phoneme Reconstruction Modeling (IPRM), predicts phoneme sequences directly from landmark trajectories and aligns them with audio-derived phonemes to reveal cross-modal mismatches. Evaluated on FaceForensics++ and DFDC, the proposed framework achieves a mean AUC of 0.967 and 0.943, respectively, surpassing single-module baselines (SDEM AUC = 0.923, IPRM AUC = 0.887) and competing deep architectures such as EfficientNet (AUC = 0.999) while maintaining interpretability through physiolinguistic cues. Experiments further demonstrate resilience against compression, occlusion, and adversarial perturbations. Limitations include reduced accuracy on extremely low-resolution videos and reliance on precise facial and audio segmentation. This research establishes a reproducible, interpretable pathway toward physiolinguistically grounded deepfake detection, providing both methodological novelty and practical forensic utility.