Interactive Sudoku Solver

Table of Contents

• Introduction
• Features
• Objectives
• System Architecture
• Installation
• Usage
• Dependencies
• Hardware Requirements
• Results
  • CNN vs. OCR
  • Hardware Performance
  • User Feedback
  • Limitations and Challenges
• Future Work
• Contributing
• License
• Contact

Introduction

The Interactive Sudoku Solver is a comprehensive tool designed to recognize, solve, and interact with Sudoku puzzles using Raspberry Pi hardware. It allows users to input puzzles manually via a touch interface or capture them using a camera. The system processes the input, identifies the puzzle structure, solves it, and displays the solution interactively. This project integrates image processing, machine learning, software engineering, and hardware interfacing to provide a seamless user experience. You can watch the video below to learn more information.

Features

• Sudoku Recognition: Detects and extracts Sudoku puzzles from images, supporting both printed and handwritten digits.
• Efficient Solving: Utilizes a backtracking algorithm to solve puzzles accurately and efficiently.
• Graphical User Interface (GUI): Intuitive interface developed for PiTFT with RPI, featuring touch interactions and color-coded elements.
• User Interaction Modes:
  • Edit Mode: Allows users to modify the puzzle.
  • Self-Solve Mode: Enables manual solving or provides hints.
• Hardware Integration: Incorporates Pi Camera, Pi TFT display, push buttons, and LED indicators for enhanced interactivity.

Objectives

The primary objective of the Interactive Sudoku Solver is to develop a user-friendly application capable of:

• Recognizing printed and handwritten Sudoku puzzles.
• Solving puzzles efficiently.
• Providing an intuitive graphical interface for user interaction.
• Leveraging computer vision, machine learning, and hardware interfacing to enhance the traditional Sudoku-solving experience.

System Architecture

The system architecture consists of the following components:

• Python Modules: Core logical units handling image processing, digit recognition, solving algorithms, and GUI management.

• Hardware Components: Raspberry Pi Camera, Pi TFT display, push buttons, and LED indicators.

• Functional Processes: Image capturing, Sudoku recognition, puzzle solving, and user interface management.

  

Installation

Prerequisites

• Raspberry Pi 4 Model B with Raspberry Pi OS installed
• Raspberry Pi Camera 3
• Adafruit PiTFT Plus 320x240 2.8" TFT + Capacitive Touchscreen
• Python 3.7

Steps

1. Clone the Repository

   git clone https://github.com/yourusername/interactive-sudoku-solver.git
   cd interactive-sudoku-solver

2. Install Python Dependencies

   pip install -r requirements.txt

3. Import Required Packages

   The project imports the following packages for handwriting recognition and PiTFT control. These packages are included within the project directories.

   • Pigame: A package for handwriting recognition.
     • Repository: n4archive/pigame
     • Included Folder: pigame
   • Pitft Touchscreen: A package for controlling the PiTFT touchscreen.
     • Repository: n4archive/pitft_touchscreen
     • Included Folder: pitft_touchscreen

   Note: Ensure that these folders (pigame and pitft_touchscreen) are present in the project directory. If not, you can clone them separately:

   git clone https://github.com/n4archive/pigame.git
   git clone https://github.com/n4archive/pitft_touchscreen.git

4. Set Up Hardware

   • Connect the Pi Camera to the Raspberry Pi.
   • Attach the PiTFT display following Adafruit's setup guide.
   • Connect push buttons and LEDs to the GPIO pins as per the parts list.

5. Configure GPIO Pins

   • Ensure the GPIO pins are correctly set up in the main configuration file.

6. CNN Model

   model.py is our model architecture file. The model file is a pre-trained Convolutional Neural Network (CNN) model, trained based on the MNIST database. Users can use it directly without training the model.

Usage

1. Launch the Application

   python main.py

2. Main Menu

   • Scan Puzzle: Capture an image of a Sudoku puzzle using the Pi Camera.
   • Select Puzzle: Choose a random puzzle from predefined difficulty levels (Easy, Medium, Hard).

3. Solving the Puzzle

   • After input, the system processes and solves the puzzle.
   • The solution is displayed interactively on the GUI.

4. User Interaction

   • Edit Mode: Modify the puzzle manually.
   • Self-Solve Mode: Solve the puzzle manually or request hints.
   • Buttons:
     • Quit (BAILOUT): Exit the application or return to the previous menu.
     • Hint: Reveal correct numbers or submit the puzzle for solving.
     • Reverse: Undo the last move or clear a selected cell.
     • Reveal: Display the complete solution.

Dependencies

• Python Libraries:
  • OpenCV (cv2)
  • Picamera2
  • Pygame
  • NumPy
  • PyTorch
  • Other dependencies listed in requirements.txt

Hardware Requirements

Part
Raspberry Pi
Raspberry Pi Camera V2
Adafruit PiTFT Plus 320x240 2.8" TFT Touchscreen
Resistors
LEDs

Results

CNN vs. OCR

• OCR (Tesseract): Achieved an accuracy of ~58.7% in digit recognition.
• CNN-Based Approach: Achieved an accuracy of ~98.3% after 14 epochs of training.
• Conclusion: CNN significantly outperforms OCR in recognizing Sudoku digits, especially handwritten ones.

Hardware Performance

• Initial Usage: Stable performance with no significant lag during image processing or puzzle-solving tasks.
• Prolonged Usage: Raspberry Pi overheated, causing performance degradation.
• Mitigation: Implemented heat sinks and improved ventilation, partially alleviating the issue.
• Future Focus: Optimize code for better thermal performance and explore more efficient algorithms.

User Feedback

• Positive: High satisfaction with ease of use, reliability, intuitive GUI, and responsive touch interface.
• Enhancements Based on Feedback:
  • Integrated LED indicators for immediate feedback.
  • Added functionality to display total steps and solving time.
  • Planned improvements include customizable themes and interactive tutorials.

Limitations and Challenges

• Image Quality Dependence: Recognition accuracy is affected by poor lighting or distorted images.
• Puzzle Size: Currently limited to 9x9 Sudoku puzzles.
• Future Improvements: Enhance image preprocessing techniques and expand solver capabilities for different puzzle sizes.

Future Work

• Advanced Machine Learning: Incorporate more sophisticated models to improve digit recognition accuracy.
• Mobile Integration: Develop a mobile version leveraging smartphone cameras for puzzle input.
• Additional Features: Introduce puzzle difficulty adjustment, step-by-step solving explanations, and customizable GUI themes.
• Performance Optimization: Enhance application performance to handle higher-resolution images and more complex puzzles.
• Expanded Puzzle Support: Extend support to larger Sudoku variants (e.g., 16x16 grids).

Team Members:

• Haixi Zhang ([email protected])
• Shuchang Wen ([email protected])

References

1. Picamera Documentation
2. Pigame GitHub Repository
3. Pitft Touchscreen GitHub Repository
4. OpenCV Tutorials
5. GeeksforGeeks: How to Use MNIST Dataset