Georgia (Ning-Yuan) Liu

she/her

Posts by Collection

portfolio

publications

A multi-dimensional index of evaluating systems thinking skills from textual data

Published in Systems Research and Behavioral Science, 2024

I developed a quantitative framework to assess systems thinking (ST) skills from text, addressing the challenge of evaluating ST’s multi-dimensional nature. Using student data, we built an integrated ST index combining statistical and optimization methods (PCA, DEA, bootstrapping). The model helps identify how factors like math background and international experience relate to ST skill development.

Recommended citation: Liu, N. G., Mahmoudi, H., Triantis, K., & Ghaffarzadegan, N. (2024). A multi‐dimensional index of evaluating systems thinking skills from textual data. Systems Research and Behavioral Science. https://doi.org/10.1002/sres.3033
Paper

The Relationships among Workload, Automation Reliance, and Human Errors in Safety-Critical Monitoring Roles

Published in Safety Science, 2024

Using real-world data from Railway Traffic Control Centers, I studied how workload and automation interact to impact human errors. We found that errors are minimized when tasks are either mostly manual or mostly automated—highlighting the risks of mid-level automation. This work offers actionable insights for managing safety in complex socio-technical systems.

Recommended citation: Liu, N. G., Triantis, K., Madsen, P., & Roets, B. (2025). The Relationships among Workload, Automation Reliance, and Human Errors in Safety-Critical Monitoring Roles. Safety Science, 185, 106775. https://doi.org/10.1016/j.ssci.2024.106775
Paper

Measuring Similarity in Causal Graphs: A Framework for Semantic and Structural Analysis

Published in arXiv preprint arXiv:2503.11046, 2025

I built a benchmark for evaluating LLM-generated causal graphs: I reviewed 40+ semantic and structural similarity metrics (NLP and graph kernel methods), selected 9, and validated them on 2,000 LLM-generated graphs. The benchmark compares causal graphs on both structural differences and the meaning of variable names. Using AI-generated synthetic data, we simulated how different people might map the same system. Our findings highlight key trade-offs between structural and semantic metrics, paving the way for better tools to interpret both human- and AI-generated models.

Recommended citation: Liu, N. Y. G., Yang, F., & Jalali, M. S. (2025). Measuring Similarity in Causal Graphs: A Framework for Semantic and Structural Analysis. arXiv preprint arXiv:2503.11046
Paper | Code

Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniques

Published in arXiv preprint arXiv:2503.21798, 2025

I developed and tested prompting strategies for translating dynamic hypotheses (text) into causal loop diagrams (CLDs) using large language models (LLMs). With curated prompting, LLMs can generate CLDs comparable in quality to expert-built ones, cutting expert analysis time 10x and lowering the barrier for novice system dynamicists.

Recommended citation: Liu, N. Y. G., & Keith, D. R. (2025). Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniques. arXiv preprint arXiv:2503.21798
Paper

Workload Dynamics in Safety-Critical Monitoring Roles: Evidence from the Belgian Railway Network

Published in Theoretical Issues in Ergonomics Science, 2025

I developed a dynamic simulation model to understand how workload impacts mental fatigue in railway traffic controllers. Using real-world data from the Belgian railway system, the model identifies a “comfort zone” for workload and shows how fatigue builds up when operators are over- or underloaded. These insights can inform scheduling, policy, and safety planning in automated transportation environments.

Recommended citation: Liu, N. G., Triantis, K., & Roets, B. (2025). Workload dynamics in safety-critical monitoring roles: Evidence from the Belgian railway network. Theoretical Issues in Ergonomics Science, 26(5), 630–653. https://doi.org/10.1080/1463922X.2025.2506424
Paper

talks

A Dynamic Model on Organizational Learning and Forgetting based on “Serious” Errors

Published:

I presented the third paper of my dissertation in the 2024 international system dynamics conference! In this presentation I introduce a dynamic model theorizing how organizations transition from a non-safety focus to a safety-focus following “serious” errors. Extending from the concept of the oscillation cycle of Organizational learning and forgetting, our model shows that time delay, organizational culture and safety threshold within an organization can have an lasting impact on safety outcomes in the organizations.

Code

teaching

Facility Planning and Logistics

Undergraduate course, Virginia Tech, Department of Industrial and Systems Engineering, 2021

I was an instructor for the undergraduate course Facility Planning and Logistics. In this course, I taught theory, concepts, and methods for designing and analyzing facilities and material flow in manufacturing, storage, and distribution environments. Topic areas include material handling systems, facility layout, facility location, warehousing, distribution, logistics, and transportation