This course teaches advanced topics in Software Engineering revolving around the following four axes: (a) Software Testing, (b) Distributed Software Systems, (c) Software Reuse, (d) Software Project Management. The first axis analyses basic and advanced concepts of software testing, starting from basic behavioral properties and ending with techniques and methods for analysis and testing of specifications and code using structural coverage criteria, as well as data and control dependence graphs. The second axis is concentrated on design and implementation issues of distributed software systems, focusing on distributed, layered architectures, middleware and provision of software as a service. Topics in the third axis cover modern software development approaches based on reusability, describing how components and open source software may be reused and integrated, and discussing their advantages and limitations. Finally, the fourth axis emphasizes on software project management issues and particularly on human resources management and cost estimation.

Course Description

The course delves into the principles of designing and implementing distributed and networked operating systems for Machine Learning (ML). It emphasizes classical operating system design principles and describes the implementation of modern systems that form the backbone of big data processing and storage infrastructure in cloud computing.

The course content includes the following topics: Virtual Memory, Process Management, and Inter-process Communication. The Unix operating system. Log-structured file systems. Distributed file systems. Virtual Machines. Topics covering the lifecycle of data analysis for ML, including data sourcing and preparation for ML, ML programming models, scalable model creation systems, and emerging topics such as ML system governance, explainability, and ethics (fairness, transparency, etc.).

Learning Outcomes

By the end of the course, students will be able to:

1. Describe the major design principles of distributed systems.
2. Understand and implement key techniques for large-scale distributed systems.
3. Comprehend the functioning of cloud computing and ML infrastructure from companies like Google, Amazon, and Facebook.
4. Read and understand research papers on computer systems engineering.
5. Implement and modify complex large-scale distributed systems for Machine Learning and evaluate their performance.

Suggested Reading

• Research articles available on Moodle (main reading material)
• Optional: Operating Systems Concepts book slides (http://codex.cs.yale.edu/avi/os-book/OS9/slide-dir/)
• Data Systems for Machine Learning lectures by Arun Kumar (https://cseweb.ucsd.edu/classes/wi24/cse234-a/schedule.html).

Course Schedule

• Week 1: Process Management. Memory Management
• Week 2: The Unix Operating System. File Systems: LFS
• Week 3: Machine Virtualization: Xen
• Week 4: Infrastructures for Big Data (Google): The Google File System, Google BigTable, Google Borg
• Week 5: ML Lifecycle Overview
• Week 6: Classical ML Training at Scale: Parameter Server, XGBoost
• Weeks 7-8: Deep Learning Systems: TensorFlow, TVM
• Weeks 9-10: System-assisted Feature Engineering and Model Selection: Cerebro, MSMS
• Week 11: Data Sourcing and Preparation: TFDV, Snorkel
• Weeks 12-13: ML Deployment: Clipper Online Prediction Serving System, Federated Learning

Grading

1. Research Paper Summaries (40%): Students must submit a summary of the research article discussed before each class. Summaries are graded on a 1-5 scale based on adherence to the guidelines on Moodle.

2. Research Paper Presentations (20%): Groups of two students will prepare a 60-minute presentation on the discussed paper. Presentations should be thorough (30-40 slides) and encourage questions and discussion.  

3. Final Exam (40%): Closed-book exam on all taught material with internet access only to download the research articles and related notes. Generative AI in any form (including more advanced search tools like Perplexity) are prohibited. The exam answers must be handwritten.

Notes for students

• Undergraduate students: They must present the research article in groups of three. Their final exam is graded more leniently than for graduate students.
• Graduate students who have already taken the undergraduate version in a previous year: They must present two research articles in groups of two. Each presentation will account for 20% of the total grade. Additionally, they must submit and present, using 10 slides, a semester project involving the use, modification, and evaluation of a large-scale computational framework covered in the course. The semester project will constitute 60% of the total grade.

Passing Criteria

1. A total score of 50% or higher is required. Students must complete summaries for at least 80% of assigned papers.
2. Failure to score above 20/40% on the final exam results automatically in failing the class, regardless of the overall grade.
3. The instructor reserves the right to make minor changes to the grading criteria and to adjust grades upwards to ensure fairness.

This module is about learning from data, in order to gain useful predictions and insights. Separating signal from noise presents many computational and inferential challenges, which we approach from a perspective at the interface of computer science and statistics. Through real-world examples of wide interest, we introduce methods for: • data munging/scraping/sampling/cleaning in order to get an informative, manageable datasets; • data storage and management in order to be able to access data - especially big data - quickly and reliably during subsequent analysis; • exploratory data analysis to generate hypotheses and intuition about the data; • prediction based on statistical tools such as regression, classification, and clustering; and • communication of results through visualization, stories, and interpretable summaries. Syllabus The module comprises a theoretical part and a series of laboratory excercises. THEORETICAL PART • Introduction. What is data science? Why is it important? Who are we? • Exploratory Data Analysis. Data Types. Data Dimensions. Visual Attributes. Effective Visual Encodings. • Visualization Design Principles. Statistical Graphs. • Data Sources. Web Scraping. Data Cleanup. • Probability review. Statistical models. Parametric and nonparametric distributions. Bootstrap. • Bias and sampling. Forms of bias. Bias-variance tradeoff. • Classification and regression. • High-dimensional data. Parallel Coordinates. Glyphs. Dimensionality Reduction. PCA. SVD. MDS. • Cross-validation. Clustering. • Bayesian thinking and methods. Prior distributions, likelihood. • Simulation methods, Bayesian computation, Monte Carlo methods. • Basic machine learning. • Amazon EC2, AWS Datastore, MapReduce, MRJob • Network sampling, community detection. LABORATORY • Introduction to Python and Numpy Arrays • Web Scraping, Data cleaning, and Pandas • Viz, and Matplotlib • SciKit Learn and Statistics in Python • SciKit Learn: inference, and cross-validation • SciKit Learn: Bayesian Analysis • MCMC in Python • EC2 and MRJob • Theano and GPU-based acceleration

This course is an introduction to the modern science of complex networks (network science), which studies the common principles and mathematical laws governing the structure and dynamics of large networks such as the Internet, Social Networks, the Web, Biological Networks, Transportation Networks, etc., as well as the principles determining the efficient communication and information flow in these networks. The student will acquire a wide range of knowledge and skills, such as the use of mathematical tools for efficient analysis and prediction of the structure of real networks, up to the design of innovative algorithms for efficient data mining and information transport within large networks. The course includes the following: network theory and representation, statistical network characterization, random graphs, formation mechanisms of real networks and evolutionary models, network geometry, "Small World" networks: Social Networks and the experiment of Milgram, Internet: novel routing techniques and Power Law topologies, dynamic processes in networks: decentralized search, random walks, rumors, gossips, epidemics, applications in large Social Networks (Twitter, Facebook) and the Web, and efficient analysis of large-scale network data. Prerequisite Background: Basic probability theory.

Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modelling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP.

This course aims at students being able to:

a) To describe and classify the fundamental design principles of deep learning systems.

b) To describe and use widely accepted techniques for deep network implementation.

c) To evaluate research articles on Deep Learning.

e)  To be capapable of carrying out a full-cycle Deep Learning Project

Course Content:

•             The foundational principle of Learning Representations.

•             Word Vectors

•             Word Vectors 2 and Word Window Classification

•             Backprop and Neural Networks.

•             Dependency Parsing.

•             Sequence-to-sequence networks.

•             Machine Translation, Attention, Subword Models

•             Question Answering.

•             Natural Language Generation.

The need to store and process massive amounts of data has led to the evolution of existing database systems while a new breed of data processing systems has emerged. This course covers a spectrum of topics from core techniques in relational data management to highly-scalable data processing using parallel database systems and MapReduce. First, the course covers the basic principles in database query processing and optimization, including index structures, sort and join processing, query rewrites, and physical plan selection. Next, the course covers topics from parallel and distributed databases, including data partitioning and distributed join algorithms. Finally, the course covers scalable data processing systems such as MapReduce and NoSQL databases (column, document, and key-value stores). The course material will be drawn from textbooks as well as recent research literature. Prerequisite background: Basic database knowledge.

This course prepares students for their Masters Thesis. It is an introduction to methodologies and techniques used for conducting original research in Computer Engineering and Informations and for preparing publishable articles. The course includes a series of seminars in various state of the art research subjects and open problems in domains of interest of the department's faculty. The main goal of the course is to stimulate interest in various research areas so that students are aided in selecting their thesis topic, as well as to equip them for the application of the scientific method in addressing computing problems and in producing high quality scientific work.

Deep Learning is the most highly sought-after skill in Artificial Intelligence. In this module, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, Recurrent Neural Networks, Transformers, Regularization, Adversarial Attacks, Deep Reinforcement Learning, Interpretability, as well as bleeding-edge Stochastic foundational regards.

This course aims at students being able to:

a) To describe and classify the fundamental design principles of deep learning systems.

b) To describe and use widely accepted techniques for deep network implementation.

c) To evaluate research articles on Deep Learning.

d)  To be capable of carrying out a full-cycle Deep Learning Project

Course Content:

•             Introduction to deep learning.

•             Neural Network Basics.

•             The foundational principle of Learning Representations.

•             Attacking neural networks with Adversarial Examples

•             Adversarial Robustness.

•             Hyperparameter Tuning, Batch Normalization.

•             Stochastic Gradient Descent Training.

•             Convolutional Layers.

•             Recurrent Layers.

•             Self-Attention Layers.

•             Generative networks: Variational autoencoders; generative adversarial networks.

•             Interpretability of Neural Networks.

•             Deep Reinforcement Learning.

•             Meta-learning.

This course is an introduction to security and privacy on the World Wide Web. After providing a historical overview and examining of the main technologies and architecture of the Internet (HTML, Javascript, DOM, Frames, HTTP(S), DNS, client-side and server-side technologies), this course will focus on security and privacy issues, with its main emphasis on potential attacks and user protection mechanisms. Some of the topics that this course will explore are: Access control and Authentication, Password-based authentication, WebAuthn. Cross-domain communications, session management and cookies, Same Origin Policy (SOP), Cross-Origin Resource Sharing (CORS), Content Security Policy (CSP). Client-side attacks (XSS, CSRF, etc.), denial-of-service attacks, phishing attacks. SQL injection attacks and server-side attacks (code injection). HTTPS, SSL/TLS, and X509 certificates. DNS attacks. Privacy and anonymity on the Web, as well as user tracking and fingerprinting techniques. 

 Τhis course aims:

- To equip students with the necessary theoretical knowledge and skills around the various phases in the design and implementation of high performance digital integrated systems.

- To familiarize students with design automation tools and material description languages, through a combination of lectures, exercises and workshops.

- To offer students comprehensive knowledge of the principles of image processing.

- To enable students to design and implement calculations for the analysis, compression, transfer and segmentation of two-dimensional and multidimensional digital signal (image and video).

Upon completion of the course, students are expected to have concrete knowledge of 2D signal processing, the basic concepts and application areas of digital image processing, to understand the operation of basic multimedia image coding standards, and to be able to use their programming experience for the development of video and video editing programs. They should also be able to use image processing algorithms and appropriate techniques to solve domain-specific problems.

This course includes topics from the broader area of Software Project Management with emphasis on modern development approaches. In this context subjects revolving around the following four axes will be covered: (a) Human resources, (b) Processes, (c) Tools, and (d) Metrics. More specifically, the lectures will teach introductory, as well as specialized concepts of the following topics: (1) Human resources management, (2) Software development processes, (3) Software project management tools, (4) Metrics and measurement of the software development process, (5) Vision and scope, (6) Resources management, (7) Time scheduling and project planning, (8) Planning documentation, (9) Activities and people, (10) Project planning and support, (11) Implementation of plans and human resources leadership, (12) Project monitoring and control, (13) Re-scheduling, (14) Product engineering. Prerequisite: The students must have successfully completed a classic undergraduate course on Software Engineering.

This course focuses on the planning and scheduling of projects with emphasis in IT projects. The students are taught how to use the concepts, methods and tools of project management to execute their organizations’ strategic initiatives efficiently, effectively and reliably. Emphasis is also given to advanced topics concerning the resource-constrained project scheduling using well-accepted OR and modern computational optimization methods from the field of meta-heuristics

The course covers an introduction to the field of production planning and monitoring, and product delivery. It investigates methods and algorithms for solving basic business problems related to production systems layout design, warehouse management, material planning, personnel staffing, production scheduling, and supply chain management. Following contemporary international practices, emphasis in given to qualitative analysis of the problems that are encountered within a supply chain – from the supply of raw materials to the production of the final products and services and their delivery to customers and end-users.

Τhe course aims to:

- Develop familiarity with Discrete Fourier Series. 

- Develop the mathematical skills for using the discrete Fourier transform and familiarization with its applications. 

- Develop evaluation and critical understanding capabilities of finite and infinite (FIR and IIR) digital response filter design methods. 

The students will be able to use digital signal processing methodologies in real problems. 

The students will gain the necessary theoretical and practical knowledge about the various phases in designing and implementing high-performance digital integrated systems with design automation tools and material description languages, through a combination of lectures, exercises and workshops. 

Upon completion of the course, students are expected to have a good knowledge of 2-D signal, the basic concepts and application areas of digital image processing, to understand the operation of basic multimedia image coding standards, and to be able to use their programming experience to the development of video and video editing programs. They will also be able to use image editing algorithms and appropriate techniques to solve specific problems. 

The goals of this course are:

• To familiarize students with advanced methods for modeling signal processing systems.
• The consolidation of the theoretical background and the appropriate use of filters.
• Learn propulsion and back linear forecasting techniques.
• Learn the methods of calculating the adaptive filter output.
• Comparative evaluation of power spectrum methods.

Students will be able to apply advanced digital signal processing methodologies to the solution of a wide range of real-world problems encountered in software applications and electronic systems.