The Challenge in Modern Education
As both students and teaching assistants in the Brown CS department, we experienced firsthand the issues facing modern education. Students struggle to receive the help they need to learn effectively, complaining about long wait times, while TAs are overwhelmed with too many students.
Before coding this project, I created a survey to get an understanding of students' use of AI and views on current office hours. While expected, our research was still surprising: 79% of students report using ChatGPT before going to office hours, and 50% use it before even trying EdStem (Brown's Q&A system). Meanwhile, two-thirds of TAs report being asked the same questions repeatedly, creating an inefficient cycle that leaves everyone frustrated.
Students reported facing long waits, feeling pressured to ask questions quickly, and often receiving incomplete answers. When they can't get help, they turn to generic AI tools that don't understand their specific course context or pedagogical goals. This creates a gap between what students need and what they can access.
A Course-Specific Solution
The ATA Approach
Unlike generic AI tools, ATA (Artificial Teaching Assistant) is designed specifically for educational environments. It uses retrieval augmented generation (RAG) to access relevant course materials including syllabi, assignment specifications, code examples, and even previous EdStem discussions.
The key innovation is course adaptation - ATA doesn't just answer questions, it provides responses informed by the specific context of each course, ensuring students get help that aligns with their instructor's teaching goals and assignment requirements.
Pedagogical Design
ATA is built with a pedagogical approach that encourages learning rather than just providing answers. Instead of simply giving solutions, it guides students through their thought process in a socratic manner, asking them to explain their reasoning and helping them discover answers themselves.
This approach is reinforced through carefully crafted examples and chain-of-thought reasoning, ensuring that ATA responds in a way that promotes understanding and critical thinking while maintaining educational integrity.
Technical Implementation
Retrieval Augmented Generation
ATA uses RAG to provide context-aware responses by retrieving relevant snippets from course materials. The system draws from a comprehensive knowledge base that includes assignment specifications and requirements, ensuring that guidance aligns with specific project goals and constraints. It also incorporates course syllabi and learning objectives to maintain consistency with the instructor's pedagogical approach and expected outcomes. Additionally, ATA leverages code examples and solutions from course materials to provide concrete, relevant illustrations when helping students understand programming concepts. Finally, the system can access previous EdStem discussions and Q&A sessions, allowing it to learn from past student questions and provide insights based on common areas of confusion or interest.
AI-Powered Guidance
The system employs GPT-4 and Anthropic Claude with few-shot learning techniques to ensure responses are both helpful and pedagogically sound. We found Claude particularly effective for generating more organic and natural responses that better engage students. By training on carefully crafted examples, ATA learns to ask guiding questions rather than provide direct answers, fostering critical thinking and deeper engagement with the material. The system is designed to encourage students to explain their thought process, helping them articulate their understanding and identify gaps in their knowledge. Rather than simply giving solutions, ATA provides strategic hints and suggestions that lead students toward understanding, allowing them to experience the satisfaction of discovery while building confidence in their problem-solving abilities. Throughout all interactions, the system maintains a supportive, educational tone that creates a safe learning environment where students feel comfortable asking questions and exploring concepts.
Safety and Content Filtering
Before displaying responses to students, ATA filters both queries and responses for malicious intent, ensuring that the AI provides appropriate, educational guidance without compromising academic standards.
Real-World Impact
From Hackathon to Production
What started as a Hack@Brown 2024 project has evolved into a practical tool used across multiple Brown University courses. The transition from prototype to production use demonstrates the real need for course-specific AI assistance in education.
Supporting Students and TAs
ATA addresses the core problems we identified through its comprehensive approach to educational support. The system significantly reduces redundant questions by providing consistent, course-specific answers that draw from the same materials and perspectives that TAs would use, ensuring continuity in the learning experience. It supports overworked TAs by handling common questions and providing initial guidance, allowing human teaching assistants to focus their time and energy on more complex student needs and personalized instruction. Students benefit from an improved experience through immediate, context-aware help that's available 24/7, eliminating the frustration of long wait times and providing support exactly when they need it most. Throughout all of these benefits, ATA maintains educational integrity through its careful pedagogical design, ensuring that students receive guidance that promotes learning rather than shortcuts that undermine the educational process.
Course Integration
The system's ability to adapt to different courses makes it valuable across various subjects and teaching styles. By pulling from course-specific materials, ATA ensures that its guidance aligns with each instructor's approach and learning objectives.
Built with: OpenAI GPT-4, Anthropic Claude, Streamlit, Python, RAG, DSPy