Course title
|
Security and Privacy in AI & Data Systems
|
Course code
|
CSE 529
|
Course type
|
Compulsory
|
Level
|
Master
|
Year / Semester
|
First Year / B Semester
|
Teacher’s name
|
Panagiotis Ilia
|
ECTS
|
7
|
Lectures / week
|
3
|
Laboratories / week
|
-
|
Course purpose and objectives
|
This course focuses on the security and privacy aspects encountered during the design, implementation, and deployment of intelligent and data systems. It aims to provide students with a clear understanding of how security and privacy risks arise within AI/ML and data-driven environments, and how these risks can be addressed through appropriate architectural and design strategies. Students will develop the ability to identify and assess vulnerabilities, model potential threats, and reason critically about privacy-conscious and security-informed design decisions, with particular emphasis on risks such as data leakage, model inversion, and adversarial manipulation.
The course builds on foundational concepts such as data integrity, authentication, access control, and threat modeling, applying them to the analysis and design of trustworthy data and AI systems. It examines privacy-enhancing methods including anonymisation, differential privacy, and federated learning, with attention to their assumptions, limitations, and design trade-offs. Security and privacy concerns are addressed across multiple layers of modern infrastructures, including data pipelines, web-based platforms, and AI/ML workflows. Topics such as tracking, fingerprinting, data poisoning, membership inference, and client-side vulnerabilities are discussed, alongside system-level principles like defense-in-depth and input sanitization. The course also introduces secure multiparty computation as a conceptual extension of privacy-by-design, and considers how architectural decisions impact system resilience, transparency, and alignment with privacy expectations and regulatory frameworks such as the GDPR.
|
Learning outcomes
|
By the end of the course, students will be able to:
- Explain the core principles of security and privacy, including confidentiality, integrity, availability, authentication, access control, and the distinction between privacy and security.
- Identify and analyze vulnerabilities and threats, applying threat modeling to assess risks across different types of architectures and system contexts.
- Evaluate privacy-preserving approaches, including anonymisation, pseudonymisation, differential privacy, federated learning, secure multiparty computation, as well as understanding their assumptions, limitations, and applicability to system design.
- Assess security and privacy risks in intelligent systems, including adversarial attacks, data poisoning, membership inference, and model inversion.
- Analyze security, privacy and data exposure risks in web-based infrastructures, including tracking, fingerprinting, and client-side vulnerabilities.
- Apply system-level design principles, such as defense in depth, sanitization, and privacy by design, to develop secure and privacy-aware architectures and reason about trade-offs in performance, resilience, fairness, and transparency.
|
Prerequisites
|
CSE 423 - Introduction to Cryptography and Computer Security
|
Required
|
-
|
Course content
|
The course content is organized into five core modules, each covering a distinct set of technical areas and concepts.
- Module 1: Core Concepts in Security and Privacy
Confidentiality, integrity, availability (CIA); authentication, authorisation, and access control; privacy vs. security; attacker models; introduction to threat modeling using STRIDE and LINDDUN.
- Module 2: Data Protection Techniques and Limitations
Anonymisation, pseudonymisation, and re-identification risks; differential privacy and federated learning; secure multiparty computation and homomorphic encryption as conceptual models.
- Module 3: Security and Privacy in Intelligent Systems
Adversarial attacks, data poisoning, and backdoors; membership inference and model inversion; mitigation strategies; privacy and explainability trade-offs in AI/ML systems.
- Module 4: Web-Based Privacy and Exposure Risks
Data collection on the web; browser security models (Same-Origin Policy, CSP, CORS); tracking, fingerprinting, and client-side vulnerabilities; implications for system design.
- Module 5: Secure System Design and Trade-offs
Defense in depth, input sanitization, data minimization, security and privacy by design; logging and retention policies; system resilience, fairness, transparency, and design alignment with GDPR and ethical considerations.
|
Teaching methodology
|
The course will be delivered through weekly lectures given by the instructor, using material specifically prepared for this course along with selected complementary resources. Lectures will cover the topics outlined in the five course modules, introducing students to relevant concepts, techniques, and design considerations. Each lecture will also include guided discussions and questions designed to encourage active participation and deepen students’ understanding of the presented concepts. After each lecture, students will be expected to prepare a short written summary of the material covered. A short discussion based on these summaries will take place at the beginning of the following lecture, to reinforce understanding and provide an opportunity for clarification and reflection.
In the final two weeks of the course, students will select and critically review a research paper from a curated list of papers related to the course topics. Each student will present their chosen paper and lead a short discussion session, with emphasis on identifying key contributions, limitations, and connections to the topics covered throughout the course. This activity will introduce students to current research work and help them build skills in analysis, critical reflection, and technical communication, while also encouraging them to draw connections across topics, assess the relevance of academic work to real-world challenges, and articulate informed perspectives on emerging issues in the field.
|
Bibliography
|
- Instructor-prepared lecture slides and notes, provided weekly in alignment with the course modules. These materials form the primary reference for lecture content and are intended to support students in preparing their written lecture summaries and in-class discussions.
- Textbooks and Supplementary Materials
No single textbook covers all topics addressed in this course. Students are expected to consult selected chapters from the following recommended books, as relevant to each module:
- Stallings, W., Brown, L., Computer Security: Principles and Practice. 4th ed., Pearson, 2017.
- Hoffman, A. Web Application Security: Exploitation and Countermeasures for Modern Web Applications. 1st ed., O’Reilly Media, 2020.
- Chio, C., Freeman, D. Machine Learning and Security: Protecting Systems with Data and Algorithms. 1st ed., O’Reilly Media, 2018.
- A curated list of research papers will be provided, covering core topics and recent developments in the field. These include work on adversarial machine learning, privacy-preserving techniques, and web security vulnerabilities among others. Students will be required to critically review and present one paper during the final 2 weeks of the course, linking its content to the topics and concepts covered throughout the semester.
|
Assessment
|
Assessment Type
|
Grade Percentage
|
Lecture Summaries
|
10%
|
Review & Presentation of Research Papers
|
15%
|
Project
|
25%
|
Final Exam
|
50%
|
|