Skip to content

AI Research Hubs

These are areas of concentration of research in AI at the USC Viterbi School of Engineering.  Each area highlights major research directions, publications, and open software or datasets. More information can be found in People and Publications.

Knowledgeable AI 

Our research is empowering AI systems with knowledge about the world to enable new capabilities.  AI systems with knowledge of commonsense and emotions have better interactions with people.  Knowledge graphs can capture how millions of entities are related, improving the ability of AI systems to understand complex domains.  The ability of AI systems to extract knowledge from diverse channels and sources allows AI systems to better understand dynamic environments.  When different sources present contradictory information, or even misinformation, AI systems need to resolve inconsistencies and decide what information to trust.  Knowledge about priorities and constraints enables better decision making and optimization.

Research:

  • Commonsense knowledge graphs: integrated knowledge graphs that include multiple sources of commonsense knowledge
  • Commonsense psychology: Large-scale logical formalization of foundational theories of human thinking, explanations, planning, and emotions.
  • Narrative interpretation: Automated interpretation of time-series data integrating perception, logical abduction, and narrative text generation
  • Evaluation of commonsense reasoning: Developed the widely-used Choice of Plausible Alternatives (COPA) benchmark for textual,  statistical, and neural approaches.

Publications:

Software and datasets:

  • CommonGen, a constrained text generation benchmark for generative commonsense reasoning
  • COPA, a benchmark evaluation for commonsense causal reasoning
  • EtcAbductionPy, an implementation of first-order probabilistic abduction in Python
  • DINE, Data-driven interactive narrative on the web

 

Additional relevant work is described in the Knowledge Graphs section.<add link>

Research:

    • Machine learning for information extraction  D. Bikel, R. Schwartz, and I developed the first machine learning approach that rivaled human performance in detecting and categorizing names in diverse human language text and speech.
    • Training data for semantic modeling In OntoNotes, my team created semantic and syntactic training and test data for parsing, semantic role labeling, linguistic coreference, and verb semantics in Arabic, Chinese, and English — all with high inter-annotator agreement.
    • Machine learning for information extraction  Scott Miller led us in developing a statistical approach to jointly extract entities and relations among them from text. 
  • Identification of visual misinformation — Wael AbdAlmageed and his team developed various technologies for identifying visual misinformation, including two generations of state of the art deepfake detection artificial intelligence algorithms and software and image manipulation and repurposing detection.
  • Organization and use of multimedia extraction output into a form useful to downstream analytics
  • Discover and represent alternative reporting of events in a knowledge base

 

Publications:

 

Datasets:

  • OntoNotes Release 5.0, https://catalog.ldc.upenn.edu/LDC2013T19, semantic and syntactic training and test data for parsing, semantic role labeling, linguistic coreference, and verb semantics in Arabic, Chinese, and English — all with high inter-annotator agreement.

 

Research:

    • Large scale federated KGs architecture and pipelines to create domain/application specific extensions of Wikidata using structured data and other KGs (Datamart)
  • Tools for creating KGs including Karma for mapping databases to ontologies and T2WML for mapping spreadsheets and web tables to KGs
    • Domain-specific KGs including the human-trafficking KG used by law enforcement
    • Ontology-Based Access for Knowledge Graphs: as part of our work integrating and querying the contents of knowledge graphs, we have started developing new ways for accessing data in a structured manner given an ontology.
  • Educational Knowledge Graphs: automatically built knowledge graph of ~12K educational materials (videos) with associated descriptions and linked authors and organizations.

Publications:

  • Szekely P. et al. (2015) Building and Using a Knowledge Graph to Combat Human Trafficking. In: Arenas M. et al. (eds) The Semantic Web – ISWC 2015. ISWC 2015. Lecture Notes in Computer Science, vol 9367. Springer, Cham. https://doi.org/10.1007/978-3-319-25010-6_12
  • Szekely P. et al. (2013) Connecting the Smithsonian American Art Museum to the Linked Data Cloud. In: Cimiano P., Corcho O., Presutti V., Hollink L., Rudolph S. (eds) The Semantic Web: Semantics and Big Data. ESWC 2013. Lecture Notes in Computer Science, vol 7882. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38288-8_40

Software:

  • Karma: A data integration tool
  • T2WML: A cell based language to map tables into Wikidata records
  • KGTK: Knowledge graph toolkit
  • Datamart: Infrastructure for transforming raw data into model-ready data
  • OBA: An ontology-based framework to create REST APIs for knowledge graphs

 

Research:

    • Decision-focused learning: integrating downstream combinatorial decision making in the machine learning pipeline through differentiable optimization: LP, Submodular Optimization, MILP or Clustering as a layer in neural networks 
    • ML-driven Graph Optimization: leveraging machine learning in combinatorial algorithms for graphs to produce novel approaches: deep reinforcement learning for graph heuristic search and ClusterNet (clustering as a layer) 
    • ML-driven tree search in MIP: improving the state of the art in exact tree search based algorithms through ML integration for Mixed Integer Programming (and recently Planning)
    • Structure in Combinatorial Optimization: Unified two schools of thought and developed the state-of-the-art computational framework for exploiting structure in combinatorial optimization problems.
    • Solving constraint satisfaction and optimization problems: CSPs and COPs can model complex decision problems, and we thus develop fast solvers for them
    • Probabilistic Planning with Realistic Preference Models: Human decision makers often have complex preference models, and we thus build planning systems that adopt these objective functions to be helpful for their users
  • Online decision making with limited feedback: understand the fundamental trade-off between exploration and exploitation in various online decision making problems such as (contextual) bandits and reinforcement learning.

Publications:

Software:

  • Open source codebases for many of the methods above <add links>

Additional relevant work is described in the Knowledge Graphs section.<add link>

 

Research:

  • Knowledge graph construction: Combining machine learning outputs with semantically-driven probabilistic models to produce reliable knowledge
  • Entity resolution in KGs: General framework that performs diverse entity resolution tasks while exploiting semantic relationships in the knowledge graph. 
  • Collective online inference: Updating reasoning with new data in highly relational settings

Publications:

Software:

  • Knowledge Graph Identification: Combine probabilistic models of semantics with noisy extracted data to make clean knowledge graphs
  • Table Understanding: Extract and identify cell information, higher-level blocks of data, and relationships between them
  • Online PSL: Adapt Probabilistic Soft Logic for online inference in response to new evidence
  • Causal Structure Discovery: Identify causal structure using probabilistic causal models

 

 

Additional relevant work is described in the Knowledge Graphs section.<add link>

Research:

  • Acquiring task knowledge: learning through tutorial instruction and through demonstrations, task-centered collaboration frameworks.
  • Semantic workflows for representation and reasoning about processes and plans, which customize steps to the given inputs, explore alternative steps, validate them, and parallelize their execution.
  • Trust and provenance, leading to the W3C PROV standard to capture the steps followed to generate a result , which supports transparency, reproducibility, and ultimately trust.
  • Automated hypothesis-driven science that includes reasoning about scientific questions as graphs, matched with lines of inquiry that map them to data queries and analytic workflows, and with meta-workflows that combine analytic results. Used in cancer multi-omics and neuroscience.
  • Scientific knowledge capture: capturing scientific software metadata, scientific hypotheses and context around scientific research (Research Objects)
  • Collaborative vocabulary development: I worked with scientists in neuro image science and geosciences to help them converge in common vocabularies to describe their data.

Publications:

Software:

  • WINGS, a workflow system with semantic constraints about data and computing steps
  • DISK, an automated scientific discovery and data analysis system
  • Organic Data Curation, a framework for controlled crowdsourcing of vocabulary standards
  • MINT: A framework for model integration with a catalog of scientific software models described semantically.
  • SOMEF:A software metadata extraction framework to automatically create structured descriptions from readme files in GitHub

 

 

  • Detecting social media manipulation: we developed a suite of advanced machine learning techniques to identify social media abuse (bots, trolls, etc.) leveraging content, metadata, temporal and network signatures characterizing machine and human behavioral patterns
  • Detecting misinformation dynamics: we developed systems to detect misinformation based on cognitive, emotional and behavioral characteristics of content diffusion and its audience

Publications:

Software:

 

Research:

  • Mining massive social graphs & streams: we developed techniques to capture, represent and analyze massive social graphs, including social streams, using knowledge representation and clustering techniques based on graph and tensor embeddings methods
  • Learning from temporal knowledge graphs

Publications:

  • Ghasemian, Amir, Homa Hosseinmardi, Aram Galstyan, Edoardo M. Airoldi, and Aaron Clauset. "Stacking models for nearly optimal link prediction in complex networks." Proceedings of the National Academy of Sciences 117, no. 38 (2020): 23393-23400.
  • Abu-El-Haija, S., Perozzi, B., Kapoor, A., Alipourfard, N., Lerman, K., Harutyunyan, H., Steeg, G.V. and Galstyan, A., 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. ICML 2019 arXiv preprint arXiv:1905.00067.
  • Elan Sopher Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi, Greg Ver Steeg, Aram Galstyan. Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning. ICLR 2021

 Software:

Datasets:

 

Research:

    • Affect in decision making: We develop theoretical and computational models of the antecedents and consequences of emotion for how individuals make decisions and cope with felt emotions.
    • Affective signaling: We develop theoretical and computational models of how expressions of emotion impact the decision-making of observers.
    • Affect understanding: We develop machine learning approaches to infer mental state and predict likely decisions from patterns of emotional expressions in context.
  • Negotiation and conflict resolution: We develop algorithms that can engage in negotiations with human users including techniques to support theory of mind reasoning and promote trust.

Publications:

  • Marsella, Stacy, and Jonathan Gratch. 2009. "EMA: A process model of appraisal dynamics."  Journal of Cognitive Systems Research 10 (1):70-90.
  • Antos, Dimitrios, Celso de Melo, Jonathan Gratch, and Barbara Grosz. 2011. "The Influence of Emotion Expression on Perceptions of Trustworthiness in Negotiation."  Proceedings of the  AAAI Conference on Artificial Intelligence.
  • Lei, S., and J. Gratch. 2019. "Smiles Signal Surprise in a Social Dilemma." 2019 8th International CoMell, Johnathan, Gale Lucas, Sharon Mozgai, and Jonathan Gratch. 2020. "The Effects of Experience on Deception in Human-Agent Negotiation."  Journal of Artificial Intelligence Research 68:633-660nference on Affective Computing and Intelligent Interaction (ACII), 3-6 Sept. 2019

Software:

 

Research:

  • <add>

Publications:

  • <add>

Software:

  • <add>

 

 

Responsible AI

To act responsibly, AI systems need to designed to guide their behavior with fairness and unbiased, to engender trust in its users, to act ethically, to provide explanations of their behavior, and to be aware of the diversity needs of people with different abilities, backgrounds, interests, and cultures.  

Research:

  • Personalized explanations: Explanations for reasoning-based predictions that have human-centric defaults and can be customized to user preferences.
  • Explainable AI for transparent agent models that can both be informed by human knowledge and be more informative to people as to the knowledge contained in those models

Publications:

Software:

 

Research:

  • Identifying and Measuring Bias in Online Social Networks: Social processes have many factors, both observed and unobserved, that need to be considered to make accurate inferences. We have developed techniques to identify, and measure the extent of different biases in a social dataset.
  • Fairness in Machine Learning Machines amplify the biases presented in their training data. We attempt to identify the sources of these biases, and to correct them. We have developed techniques for network analysis to incorporate marginalized voices in the network.

Publications:

  • Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019). A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635.
  • Pfeffer, Jürgen, Katja Mayer, and Fred Morstatter. "Tampering with Twitter’s sample API." EPJ Data Science 7.1 (2018): 50. [DOI] [PDF]
  • Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. M. (2013, June). Is the sample good enough? Comparing data from Twitter's streaming API with Twitter's firehose. In Seventh international AAAI conference on weblogs and social media. [DOI] [PDF]
  • Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2017). Feature selection: A data perspective. ACM Computing Surveys (CSUR), 50(6), 1-45. [DOI] [PDF]
  • Preference-Informed Fairness. Michael P. Kim, Aleksandra Korolova, Guy N. Rothblum, Gal Yona. FAT* 2020. ITCS 2020.
  • Discrimination through optimization: How Facebook's ad delivery can lead to biased outcomes. Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, Aaron Rieke. CSCW 2019. 
  • Moyer, Daniel, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. "Invariant representations without adversarial training." Advances in Neural Information Processing Systems 31 (2018): 9084-9093.
  • Gupta, Umang, Aaron Ferber, Bistra Dilkina, and Greg Ver Steeg. "Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation." AAAI 2021

 

Research: AI and Machine Learning while preserving a rigorous notion of privacy, differential privacy.

Publications:

 

 

Interactive and Collaborative AI

Effective collaboration requires understanding language, communicating ideas, working on shared or related goals, learning from people, and taking into account the preferences and priorities of different users.

Research:

  • Abstract Meaning Representation Defined an annotation standard for whole-sentence semantic annotation; built a large corpus and sophisticated annotation tool, developed software for automatically parsing English into AMR and vice versa, conducted two shared tasks.
  • Transfer Learning for Low-Resource Machine Translation Developed now-standard technique for building low-resource neural machine translation systems by leveraging higher resource parallel data.

Significant Publications:

 

Software:

  • uroman: A universal romanizer that converts nearly all orthography systems into a common format readable by English speakers. 
  • SARAL: A low-resource cross-lingual domain-focused information retrieval system for effective rapid document triage.
  • English-to-AMR parser: Uses machine translation techniques to semantically parse English into AMR semantic graph format.

 

 

Research:

  • Social Simulation through the application of decision-theoretic AI algorithms, in combination with recursive models for Theory of Mind, to modeling human social behavior (decision-making, social influence, cognitive appraisals) for descriptive and prescriptive purposes
  • Data-driven modeling to automatically build decision-theoretic models of human behavior from data via search that is constrained by available knowledge provided by experts
  • Team-oriented programming to leverage domain-independent algorithms for teamwork in support of a more abstract language for specifying the domain-specific knowledge required for an organization, potentially containing both humans and agents, to perform a hierarchical joint task
  • Plan recognition to infer the mental states (perceptions, goals, plans, etc.) of other agents (whether software or human) based on observations of their behavior

Publications:

Software:

  • PsychSim: Software implementation of multiagent social simulation using recursive decision-theoretic agents

 

Research:

  • Cooperative Auctions: In auction-based coordination systems, the agents bid on tasks and the system then assigns them tasks in real-time based on their bids
  • Multi-Agent Path Finding: Teams of agents assign target locations among themselves and then plan collision-free paths to their target locations, see mapf.info.

Publications

Software

 

 

Adaptive AI

Research:

  • Latent factor discovery: Introduced an information-theoretic principle, Total Correlation Explanation (CorEx), for unsupervised discovery of latent factors that capture interpretable multivariate factors across domains including gene expression, neuroimaging, and social science.
  • Invariant representation learning: This line of work links information-theoretic optimization and invariant representation learning, which seeks to control information present in representations to reflect domain knowledge or for excluding data bias in fair machine learning. 
  • Controlling information for efficient machine learning: New ways to control information in representations (optimal rate-distortion trade-offs via echo noise) and to control information stored in neural network weights about training data to prevent over-fitting. 

Publications:

  • Brekelmans, Rob, Vaden Masrani, Frank Wood, Greg Ver Steeg, and Aram Galstyan. "All in the exponential family: Bregman duality in thermodynamic variational inference." ICML 2020 arXiv preprint arXiv:2007.00642 (2020).
  • Brekelmans, Rob, Daniel Moyer, Aram Galstyan, and Greg Ver Steeg. "Exact rate-distortion in autoencoders via echo noise." In Advances in neural information processing systems, pp. 3889-3900. 2019.
  • Pepke, Shirley, and Greg Ver Steeg. "Comprehensive discovery of subsample gene expression components by information explanation: therapeutic implications in cancer." BMC medical genomics 10, no. 1 (2017): 1-18.
  • Gallagher, Ryan J., Kyle Reing, David Kale, and Greg Ver Steeg. "Anchored correlation explanation: Topic modeling with minimal domain knowledge." Transactions of the Association for Computational Linguistics 5 (2017): 529-542.

Software:

  • Corex_topic Using CorEx latent factor models for controllable topic discovery

 

 

Embodied AI

Research:

  • Fast replanning: we study how to build planning systems that adapt quickly to changing situations
  • Deep reinforcement learning for navigation planning: we study how to learn reactive planning systems
  • Any-angle path planning: we study versions of A* that propagate information along grid edges (to achieve small runtimes) but without constraining the paths to grid edges (to find short paths)
  • Preprocessing-based path planning: we study algorithms that first analyze a given graph in a preprocessing phase to generate auxiliary information which can then be used to significantly speed-up online shortest-path queries
  • Biologically-inspired multi-robot systems: we study how to control ant robots, which are simple robots that leave trails in the terrain

Publications:

Software:

Research:

  •  

Publications:

  •  

Software:

  •  
University of Southern California