Admin Login
Username:

Password:

Overview

In general, I am interested in artificial intelligence, cognitive science and their applications to real world problems. Currently, I am working for machine learning, pattern recognition, machine perception, computational cognitive systems and their applications to intelligent system development. I am most keen on supervising self-motivated PhD students working in these areas.

In my Machine Learning and Perception (MLP) Lab, I am offering projects grouped around the aforementioned interests of mine (listed in the alphabetic order). All of these projects are open ended and a candiate is expected to contribute to their chosen projects by drafting a preliminary proposal to demonstrate your research capability and to indicte your own interest in your PhD studies (Note: none of projects below is applicable to MPhil studies). Prospective students are expected to gain a reasonable background prior to enrolment in order to qualify for his/her PhD studies working with me, including good mathematics, machine learning and certain area/subject knowledge dependent on projects, e.g. computer vision, speech/acoustic signal processing, music theory, video game engines, psychology, cognitive science and neuroscience. Also, the good programming skill is essential for all the suggested projects. In general, a qualified candidate must be able to use Python and/or Matlab  fluently (it would be ideal that you can programeme with a high-level programming language such as C, Java and C++ as well). In addition, you should have the essential knowledge and skills on modern machine learning toolboxes/developmental tools, such as Pytorch or TensorFlow, Python-based machine learning toolkits and so on.

If one has any own project idea which falls into my expertise mentioned above, they are welcome to discuss it with me. As a result, I am very happy to consider new or further project suggestions they propose as long as those projects are suitable for PhD studies and relating to my research interests.


List of Suggested Projects


Automatic Activity Analysis, Detection and Recognition

Activity detection, analysis and recognition are related to several important areas in machine perception research and applications. Goals include automatic analysis or interpretation of ongoing events, detection of pre-specified events and their context from multimedia data, and recognition and prediction of actions of one or more agents from a series of observations. There are several applications demanding such techniques, e.g., surveillance systems, patient monitoring systems, sport training-assistant systems, and a variety of systems that involve interactions between persons and electronic devices such as human-computer interfaces. In general, these applications require recognition of high-level activities, often composed of multiple simple actions of clients, with cues from single modal, e.g., audio stream or video clips (image sequences), or information fusion from multi-modal data, e.g., different types of behaviour data collected in mobile devices. 

In general, the project aims developing effective techniques for automatic activity detection, analysis, recognition with flexibility applicable to different mono-modality, e.g., video clips or audio stream, or multi-modality by using cues from different information sources. Several issues are going to be investigated including low-level feature extraction, intermediate-level descriptors, high-level semantic analysis/representation and knowledge compression required by mobile device in real applications. In addition, temporal information processing and context-aware computing techniques are also core issues to be studied in this project. While this is a generic project description, the specific project will be well-defined with the perspective yet self-motivated student’s input. As an artefact of such a project, normally, a prototype with some clear scenarios or requirements needs to be developed to demonstrate the effectiveness and flexibility along with developing novel approaches. While the relevant fundamental research is expected to be conducted, the project is suitable for one who has a clear targeted application area in mind.

In order take this project, it is essential to have good background knowledge in both machine learning and image/speech signal processing as well as excellent programming skills.

   List of Suggested Projects


Automatic Emotion Detection, Analysis and Recognition

Affective computing is a branch of Artificial Intelligence that relates to, arises from, or deliberately influences emotion and other affective phenomena. Research in affective computing is of interdisciplinary nature, which combines computer science with many other fields, e.g., psychology, cognitive science, neuroscience, sociology, medicine, psychophysiology, ethics, and philosophy, in order to enable advances in basic understanding of affect and its role in biological agents, and across a broad range of human experience. From a human-machine interaction perspective, the most important topic in affective computing is automatic emotion detection, analysis and recognition from human behaviours including facial expression, speech and body gestures. 

In general, this project is going to investigate critical problems underlying emotional representation learning, emotional pattern discovery, emotional pattern modelling and recognition.  This is a flexible project; i.e., it could be either a fundamental research oriented project that learns a “universal” emotion representation that is insensitive to different factors or a practical project that applies state-of-the-art machine learning and signal processing techniques to the emotion detection and recognition in a real scenario. In addition, this project mainly focuses on mono-modal emotion but can also be extended to the development of multimodal affective computing techniques, i.e., fusion of different emotional information for decision making. For demonstration, a prototype normally needs to be established based on the proposed approaches for a real application, e.g., computerized tutoring in an e-learning environment. While the relevant fundamental research is expected to be conducted, the project is suitable for one who has a clear targeted application area in mind.

In order to take this project, it is essential to have good machine learning, speech/image signal processing, Psychological background knowledge on emotion theories (if working on fundamental research) as well as excellent programming skills (if working on applications).

  List of Suggested Projects


Automatic Music Generation via Deep Learning

Music appears as an art form and cultural activity whose medium is sound organized in time, which is an ultimate language for human beings. Music is performed with a vast range of instruments and vocal techniques; there are solely instrumental pieces, solely vocal pieces and pieces that combine singing and instruments. Above all, music generation (aka musical composition) is regarded as a creative task by creating a specific style of musical content or writing a new piece of music. For automatic music generation, algorithmic composition techniques have been developed for several decades. While some progresses were made, there are still many open challenges, e.g., effective music representations and modelling, for automatic music generation. Deep learning has been proven to be a powerful technique in tackling complex real-world problems. As opposed to handcrafted models, such as grammar-based or rule-based music generation, deep learning techniques allow for automatic music generation via learning a model from an arbitrary corpus. As a result, a single learning model trained on different corpora may be used for various musical genres.

The project is going to investigate and develop novel automatic music generation techniques with exploring and applying the state-of-the-art generative models such as generative adversarial networks (GAN) and variational auto-encoder (VAE) as well as deep sequence modelling techniques such as memory augmented recurrent neural networks. In this project, main issues to be studied include effective representation of music notes suitable for music content generation, modelling various music structures that effectively express the notions of harmony and melody, specific music style modelling and transfer and effective yet efficient music generation strategies to be used in deep learning and generative models. In particular, this project is suitable for one who is enthusiastic about music and interested in applying the state-of-the-art machine learning techniques in tackling complex real-world problems.

It is worth highlighting that this is an extremely challenging project of a great novelty. In order to take this project, research experience related to the application of related deep learning techniques may be required. It is also essential to be self-motivated and to have decent background knowledge in music theory, mathematics, machine learning as well as good programming skills. Apart from those stated above, it would be ideal that one has knowledge and skills in performing music, e.g., vocal or playing a musical instrument.

  List of Suggested Projects


Biologically-Plausible Continual Learning

Continual learning (aka lifelong learning) refers to a problem on how a learning system learns multiple tasks in succession over the lifespan where later tasks do not degrade the performance of the system learned for the earlier tasks and, ideally, the system can leverage the knowledge learned in previous tasks to facilitate learning the new tasks better. While human brains have such a remarkable capability to learn various tasks without negatively interference during lifelong learning, all machine learning models, deep learning models in particular, generally fail for continual learning due to the notorious “catastrophic forgetting” phenomenon. Recently, efforts have been made in addressing this issue in deep learning research but all the attempts so far are for artificial neural networks without taking biological plausibility into account. On the other hand, to a great degree, continual learning mechanisms in human brains remain unknown.

The project is going to investigate and develop biologically-plausible continual learning mechanisms based on biologically-plausible neural networks, e.g., spiking neural networks, and the existing evidence from neuroscience and cognitive science via carefully formulated hypotheses. In this project, main issues to be studied include formulating proper hypotheses, developing biologically-plausible building blocks and learning algorithms required by continual learning, experimentation to verify the formulated hypotheses and exploring the possibility of using the research outcome to inform other disciplines. Also, this project includes running real-time simulations on a neuromorphic computer, e.g., SpiNNaker, subject to its availability. In this case, how to map biologically-plausible continual learning mechanisms properly onto the neuromorphic computer is going to be studied as well. In general, this project is suitable for one who is interested in fundamental research in biologically-plausible deep learning and exploring the unknown aspects of human brains.

It is worth highlighting that this is an extremely challenging project of a great novelty. In order to take this project, it is essential to be self-motivated and to have decent background knowledge in mathematics and machine learning as well as good programming skills. It would be ideal if one has the research experience in spiking neural networks and computational cognitive modelling.

  List of Suggested Projects


Contextualised Multimedia Information Retrieval via Representation Learning

Multimedia Information Retrieval (MIR) is an important research area in AI that aims at extracting semantic information from multimedia data sources including perceivable media such as audio, image and video, indirectly perceivable sources such as text, bio-signals as well as not perceivable sources such as bio-information, stock prices, etc. In general, the main MIR tasks of MMIR can be summarization of media content as a concise description via feature extraction, filtering of media descriptions via elimination of redundancy, and categorization of media descriptions into classes to facilitate retrieval. In essence, the fundamental problem underlying all the MIR tasks is how to bridge the gap between low-level multimedia data and the semantics conveyed by such data. On the other hand, the accurate semantics are not able to be decided until the context is given as perfectly exemplified in natural language understanding. In general, we believe that the contextual information would be extremely useful in MIR if such information can be captured/modelled.

Unlike many existing researches in MIR, this project is going to investigate how to explore and exploit context information from multimedia annotation and side information sources to facilitate different MIR tasks. While there are other approaches to this problem, this project focuses on exploring the synergy between contextualised semantic representations and low-level descriptors to bridge the aforementioned gap via machine learning. The main issues to be investigated include novel multimedia feature extraction methods suitable for contextualised MIR, contextualised semantics modelling, effective media data descriptors and their joint latent representations. In particular, the aforementioned research issues would be investigated by taking real environmental factors, e.g., noise and mismatch conditions, into account. Based on the proposed approaches, a prototype of high performance for a target application would be established, e.g., personalised MIR for video stream retrieval. While the relevant fundamental research is expected to be conducted, the project is suitable for one who has a clear targeted application area in mind.

In order to take this project, it is essential to have good machine learning and multimedia (signal processing) background knowledge as well as excellent programming skills.

  List of Suggested Projects


Deep Learning for Temporal Information Processing

Temporal information processing covers a broad class of learning problems where knowledge can be acquired from data of a sequential order, e.g. speech modelling, image sequence analysis, robot navigation, financial prediction, and so on. In addition, different learning paradigms, supervised, unsupervised, and reinforcement styles, may be involved in temporal information processing. Recent studies suggested that deep learning has several advantages for temporal data analysis and information extraction regardless of a learning paradigm and can overcome a number of weaknesses of existing temporal information processing methods. In this project, the following issues will be investigated: 1) Exploration of novel deep architectures for effective encoding temporal information, 2) Model selection issues in the context of the hybrid learning strategy for different learning paradigms, 3) Highly nonlinear temporal coherence/factor analysis with different strategies, and 4) Applications in selected real world temporal information processing tasks, e.g., audio stream analysis and financial/stock data mining.  In general, this project is suitable for one who is interested in fundamental research in machine learning while it is acceptable for one who has a relevant application problem in mind and wishes to tackle their problems with an emerging technology such as deep learning.

In order to take this project, it is essential to have good background knowledge in mathematics, machine learning as well as good programming skills.

  List of Suggested Projects


Ensemble Strategies for Semi-Supervised/Unsupervised/Transfer Learning

Traditionally there are two main paradigms in machine learning, supervised vs. unsupervised learning. A supervised learning algorithm uses teacher's information (labelled examples) to train a learner while unlabelled data are automatically categorised by an unsupervised learning algorithm without using teacher's information. In reality, however, labelled examples are often difficult, expensive, and/or time-consuming to obtain, which demands the efforts of experienced human annotators, while unlabelled data may be relatively easy to collect. Semi-supervised learning offers new techniques with the use of large amount of unlabelled data along with some labelled examples. In some situations, no labelled data are available so that one can only adopt the unsupervised learning paradigm for learning. Nevertheless, a common issue for both semi-supervised and unsupervised learning paradigms is how to exploit the information conveyed in unlabelled data. In a generic sense, the aforementioned learning problems may be naturally extended to transfer learning where other information sources can be explored to facilitate the current learning task in hand.

Ensemble learning studies machine learning algorithms and architectures that build collections of learners towards achieving better performance than an individual learner. This project is going to investigate typical ensemble learning methodologies, e.g., sequential and hierarchical combination of learning models, within the semi-supervised/unsupervised/transfer learning paradigms. The  representation learning models that tend to tackle challenging real world problems that violate the standard yet conservative statistical assumptions made in the current machine learning algorithms. The main issues to be studied include theoretical/empirical investigation on novel ensemble representation learning framework including miscellaneous combination strategies in terms of generalization/stability and computational complexity, exploration/exploitation of unlabelled data or various information sources across different component learners and automatic model selection in the context of semi-supervised/unsupervised/transfer learning. In general, this project is suitable for one who is interested in fundamental research in machine learning while it is acceptable for one who already has a relevant application problem in mind and wishes to tackle their problems with an emerging technology such as ensemble learning. It is worth mentioning that this project description is generic and a specific project needs to be well-defined with a self-motivated student’s input.

In order to take this project, it is essential to have excellent mathematics and machine learning background knowledge as well as good programming skills.

  List of Suggested Projects


Information Component Analysis via Deep Learning

As their prominent characteristics, perceptual data often convey the mixing information, which often results in the inadequate performance for a specific perceptual information processing task due to the interference of irrelevant information components. For example, facial images typically convey the mixing information including identity and expressions. For specific tasks like face and facial expression recognition, the mixing information components are hardly separable, which results in difficulties in either of two tasks. The same problem also exists in speech information processing where speech conveys the mixing information including linguistic, speaker, emotional and environmental characteristics. Furthermore, there is no equal amount of information for mixing components; e.g. linguistics often overwhelmingly dominates the information in speech. The nature of perceptual data gives rise to considerable challenges in their modelling, analysis and recognition.

The project is going to investigate and develop a generic approach to information component analysis for perceptual data with state-of-the-art machine learning techniques, deep learning. Surrounding the main theme on how to disentangling/extracting information components, main issues to be studied include objective-driven high level abstraction of perceptual data in flexible representation forms, novel building blocks and deep learning models including architectures and learning algorithms to carry out an information component "filter'' and theoretic information aspects in measuring the extracted information components. For demonstration, an information component analysis prototype would be developed for a real application, e.g., speech or facial information component analysis. In general, this project is suitable for one who is interested in fundamental research in machine learning while it is acceptable for one who has a relevant application problem in mind and wishes to tackle their problems with an emerging technology such as deep learning.

It is worth highlighting that the hypotheses set in this project are original and hence this is an extremely challenging project of a great novelty.  In order take this project, thus, it is essential to be highly self-motivated and to have excellent background knowledge in mathematics, machine learning, image or speech signal processing and good programming skills.

  List of Suggested Projects


Machine Learning and Cognitive Modelling Applied to Video Games

Video games have been viewed as an ideal test bed for the study of AI. However, most of the academic work in this area focused on traditional board and card games where limited AI techniques have been tested. On the other hand, interactive vedio game development, particularly video games, has grown up to be an industry of a huge market between $35 billion and $50 billion. Interactive video games provide a forum for interaction between agent and human in cyberspace and are argued to be of educational value apart from entertainment. Recent studies revealed that most of exiting interactive games lack innovation (e.g., most of existing games have only predefined, static and predictable game agent responses) and fail to consider player satisfaction (e.g., frustration caused by failures in performing some actions), and the next generation interactive games demand improving the player experience in fantasy, innovation, curiosity, challenge and imitation of human intelligence. Thus, there is an unexplored opportunity for cognition-aware machine learning to make interactive games more interesting and realistic. Machine learning would provide a new way to improve behavioural dynamics for automatic generation and selection of behaviours, which offer opportunities to create more engaging and entertaining game-play experience. Furthermore, computational cognitive modelling techniques along with machine learning allow for modelling player/agent behaviours and creating vivid cognition-aware environments. 

This project is going to investigate machine learning and cognitive modelling techniques for developing next generation video games. The main issues in this project include autonomous learnable agents for generic video game AI, novel learning algorithms for game content space exploration and exploitation, novel cognition-aware learning algorithms for real-time adaptation mechanisms, player-experience driven automatic game content generation and player behaviour modelling as well as new game-genre framework via deploying psychological and cognitive theories. As a part of this project, normally, a prototype with an appropriate genre will be developed with the proposed learning algorithms under the new game-genre framework to demonstrate the novelty of the proposed methodology. It is worth mentioning that this project description is generic, and a specific project needs to be well-defined with a self-motivated student’s input. 

In order to take this project, it is absolutely essential (prerequisite) to have video gaming programming experience and excellent programming skills in C++ and/or other common gaming programming languages (Please refrain from making any inquiry if one does not meet this condition). In addition, it also requires good machine learning and basic cognitive science knowledge .

  List of Suggested Projects


Multi-Task Learning and Applications

In traditional machine learning, a learning system can be trained to deal with a specific single task, while human is able to complete multiple tasks with the same learning strategy. To overcome the limitations in traditional machine learning, multi-task learning techniques are demanded by for artificial general intelligence (AGI). For AGI, a learning system works for various tasks by sharing relevant knowledge between tasks so that learning a new task is done more efficiently and the learning system can generalise better on multiple tasks.

This project is going to develop novel learning systems and learning algorithms for multi-task learning in terms of different learning paradigms ranging from supervised, unsupervised to reinforcement learning and their applications to real world problems. The main research theme is how to share the generic knowledge and the representations applicable to different tasks without scarifying the previous learning outcome for a specific task. In a lifelong learning setting that a new task is learnt by a system already trained on other tasks, harmless knowledge transfer is also an unsolved issue and hard to carry out in the use of deep learning for multi-task learning due to catastrophic interference. Furthermore, this project also needs to address common issues in machine learning such as domain shift. Regarding applications, multi-model information processing, robotics and general video game playing are among the proper test beds for different learning paradigms. It is worth mentioning that this project description is generic and a specific yet well-defined project needs to be developed based on a self-motivated student's own input.

In order to take this project, it is essential to have excellent mathematics and machine learning background knowledge as well as good programming skills.

  List of Suggested Projects


Zero-Shot Learning and Applications to Multimedia Information Retrieval

Zero-shot learning refers to a novel paradigm on learning how to recognise new concepts by just having a description of them. For example, zero-shot learning works on a setting of solving a classification problem when no labelled training examples are available for all classes, which are divided into two class subsets: training and unseen classes, where there are only examples of training classes available to be used in building up a classifier. Under the zero-shot paradigm, it is expected that a classifier trained on only the training-class examples works for test data of unseen classes by exploiting the side information regarding the semantic relationship between training and unseen classes. When this learning paradigm is used in multimedia information retrieval, there is a big challenge; i.e., there is a semantic gap between raw media data and their semantic meaning. As a new learning paradigm, zero-shot learning paves a new way to address issues such as a lack of training examples in supervised learning and expand the capacity of a learning system to deal with unknown situations as same as human beings do.

The project is going to investigate effective zero-shot learning techniques as fundamental research and their applications in real world problems such as multimedia information retrieval and multi-task reinforcement learning. The main research theme is how to bridge the semantic gap between raw media data descriptions and their semantic meaning. Surrounding this main theme, main issues to be studied include media representation learning, semantic representation learning and latent embedding frameworks with the state-of-the art deep learning and representation learning methodologies. As domain shift and the nature of high-dimensional data are generic issues in zero-shot learning, it is inevitable that this project has to address the common issues appearing in transfer learning and manifold learning. Regarding the applications to multimedia information retrieval, a real scenario based on video streams will be selected to be used as a test bed, which is a non-trivial part of this project. Likewise, zero-shot knowledge transfer in general video game playing could be a test bed for zero-shot reinforcement learning. It is worth mentioning that this project description is generic and a specific yet well-defined project needs to be developed based on a self-motivated student's own input.

In order to take this project, it is essential to have excellent mathematics and machine learning background knowledge as well as good programming skills.

  List of Suggested Projects


 


This page was last updated at 04:13pm, July 4th, 2019.