MW: Creator Survey

Background

🏅 Creators Invitational 2021 (CI 2021)

The product team organized a contest called the Creators Invitational 2021, open to creators and aspiring developers. The selected contestants were tasked with creating and publishing content on MapleStory Worlds.

🛠 Creator Experience

To make MapleStory Worlds more appealing to creators, the product team broadened its focus to the creator experience, moving away from the general user and player experience. Their goal is to provide a developer-friendly environment for creators.

Objectives

1️⃣ To gain insights into players’ overall experience over time

2️⃣ To identify areas for improvement (later delve into them through further in-depth studies)

Methods

📄 Surveys (Regular Surveys)

To gain insights into players’ overall experience over time and identify areas for improvement, we conducted regular surveys.

1️⃣ Gaining insights into players’ overall experience over time

We aimed to collect such data as overall satisfaction and the perceived level of difficulty on a regular basis. We also evaluated the system’s usability with the end survey, using System Usability Scale (SUS).

2️⃣ To identify areas for improvement by the development stages

Our second objective was to understand areas for improvement among the following development stages. Through the surveys, we aimed to define the areas, we then later dived deeper into them with further studies.

👤 Respondents

A total of 308 independent creators who participated in Creators Invitational 2021 (CI 2021) were subject to the surveys. To the initial survey, 291 creators responded, 271 creators to the midpoint survey, and 122 creators to the end survey.

The majority of the respondents had experience in creating content using platforms, but most of them were new to the Lua language.

Reporting

🧠 Leveraging ML to quickly summarize the ‘why’ behind their ratings

We used both sentiment analysis and topic modeling for EDA (exploratory data analysis) to quickly summarize topics of the negative responses. It turned out most of the negative responses included #Scripting and #Resources in their comments.

Also, respondents who rated “difficult” and “very difficult” for the perceived difficulty, mentioned #Scripting and #Skill Levels for the reasons why they believed so.

🎯 Defining areas for improvement by the development stages

The study revealed that the respondents were relatively unhappy about #Scripting and #Navigating Resources throughout all stages.

Leveraging factor analysis, it also came down to a conclusion that the Resource Search feature is the most determining factor for the overall satisfaction of MapleStory Worlds. As a result, we, along with the stakeholders, reached a consensus to conduct further studies to better understand why creators are experiencing difficulties with scripting and navigating resources.

📈 (Appendix) Tracking The System Usability Scale (SUS) as a new KPI

Starting with this study, we suggested keeping track of the System Usability Scale (SUS) as a new KPI for MapleStory Worlds.

Research Impact

👩‍🏫 Post-Project Research Readouts

• • Engaged the product team in conducting additional studies:
    • Further in-depth studies about the creator experience
    • Further studies about new features such as ‘Cocreation’
    • More studies about the players (spanning from the creators)

📌 Product Impact

• • ‘Resource Search’ has been improved:
    • Sparked discussions on improving ‘Resource Search’ leveraging creators’ feedback.
  • Scripting has been improved:
    • Related features, e.g., ‘Script Editor’, ‘Property Editor’, etc., have been improved.
  • Social features have been updated:
    • The development of new out-game social features, such as ‘My Home’ is in progress.
    • Social features, e.g., friends’ online status, adding friends, etc., have been improved.
  • Others:
    • Avatar customization features, e.g., ‘Closet’, ‘Shop’, etc., have been improved.
    • The concept of the ‘Cocreation’ features has been conceived.
    • The improvement plan for the onboarding guide has been defined.

📌 Strategic & Cultural Impact

• • Research findings were shared in a WIP meeting.
  • Research findings were presented in the management report.
  • Task forces were formed with a focus on feature improvements.
  • Encouraged the product team’s cultural buy-in about UXR.
  • Helped define strategic priorities for development.

📌 Internal Impact

• • Established the first case for implementing long-term collaboration with the product team.

My Role

In the beginning, MapleStory Worlds was merely an idea. I took the lead in research, identifying key needs and opportunities, laying the groundwork for its early stages, and fostering long-term collaboration with stakeholders.

1️⃣ Leading Strategic · Tactical Research

I managed a small team of researchers over a two-year collaboration with the product team. Our efforts encompassed diverse areas, including gameplay experience, content creation, and publishing, utilizing generative research and co-creation methodologies.

2️⃣ Uncovering Needs and Opportunities

In the early stages, we explored user needs and opportunities in first-mover metaverse and social platforms like Roblox, Minecraft, and TikTok. As the project evolved, we expanded our focus to include diverse user types—regular players, creators, and streamers.

Conclusion

📌 Early Stage · Long-Term Collaboration: How it all went

Wrapping up this long-term collaboration, I gave a speech at MCI22 (MapleStory Worlds’ Creators Invitational 2022) on the topic of ‘Co-Creating MapleStory Worlds with the User’. I highlighted our extensive co-creation and collaboration with diverse stakeholders and users, sharing various studies and insights gained from them.

Research Impact

📌 Cultural Impact

Initiated during the ideation phase, the early stage collaboration defined target users through personas, uncovering their needs and opportunity areas. This collaborative design thinking process, involving designers, developers, and users—including creators and streamers—helped shape the product’s go-to-market strategy.

📌 Product Impact

This approach demonstrated how early collaboration can drive long-term product success. MapleStory World achieved a successful launch and has since become a beloved service, now with 60K+ active players, continuing to thrive.

At the conclusion of the collaboration, user needs-based heuristics were provided as a lasting KPI for the product team.

Page 44444

MCI22 강연 슬라이드_게임UX분석팀 이종윤_20221125(현장 수정)_영문화