Public Distribution System (PDS) plays a crucial role in providing basic commodities such as wheat, rice, sugar, and kerosene to the underprivileged at affordable prices. However, traditional Fair Price Shops (FPS) often face significant challenges, including irregular operating hours, long queues, and delays in supply, requiring citizens to visit frequently to check availability. In light of the COVID-19 pandemic, social distancing guidelines further complicate the situation. To address these issues, this project proposes a modernization of the PDS by implementing a virtual queuing system utilizing the Q-Learning algorithm. This innovative solution aims to replace physical queues with automated slot allocation, ensuring ration cardholders receive SMS notifications for their designated time slots to collect goods. With the addition of two re-slot allocations, users have flexibility in case of missed appointments, minimizing unnecessary trips. The system also enables users to view product details online, saving time and improving accessibility. By leveraging Q-Learning, a reinforcement learning algorithm, this approach offers a more efficient, transparent, and socially distanced method of distribution. This project not only enhances the efficiency of the PDS but also serves as a model for using technology to improve the delivery of government services, especially in times of crisis.