IBM IXN and UCL Computer Science investigate Open-Source Immersive Social Engagement during Covid-19

Three second-year undergraduate teams from UCL Computer Science (UCL CS) worked on UCL Industry Exchange Network (UCL IXN) projects together with the IBM IXN programme, to create the Franklin Immersive Social Engagement (FISE) experiments project. The eight students developed multi-platform Proof of Concepts for reducing social isolation through communication, collaboration and engagement opportunities, making use of open-source technologies and real-world service APIs.

The IBM FISE v2.0 project is now in its second year and has sought to tackle loneliness, looking at existing systems and services that could assist in the enduring crisis. In the fight against social isolation, UCL IXN aligned with IBM IXN to analyse and develop proof of concept solutions to help those in need during the pandemic. What started with three separate group projects on chatbots, augmented reality, and remote-calling management, resulted in engineering a brand new open-sourced architecture that can help software developers build further apps to help tackle social isolation.

The overall project was developed as three teams working on specific use case scenarios. One team was working on a customisable, open-source voice assistant framework ("Ask Bob") for low-power desktop devices that safeguards privacy by processing speech offline for users. This meant it could potentially be used in sensitive user groups. Meanwhile, the other two groups worked on a fully open-sourced video conferencing web app ("FISE Lounge") for the socially isolated and elderly based on Jitsi technology, and a voice-enabled concierge Android app ("FISE Concierge") making public web services more accessible and available to less technologically savvy users.

The three projects became architecturally linked into a single project under IBM's Franklin Immersive Social Engagement (FISE) open-source experiments project.

Jeremy Lo Ying Ping, the lead student architect for the three teams, stated that as with all group projects there are challenges, but more so when integrating with multiple teams. This particular project meant that organisation and delegation were of high importance, but nonetheless a challenge that allowed for success; 

“There were certainly challenges throughout the development process, both technical and non-technical. On the technical side, as the coordinator of the three teams, one of my main responsibilities was to lead on designing and coordinating the development of a 'system architecture' to link the three projects together to make them interoperable. Brainstorming creative software engineering solutions to connect the projects was certainly one of the things I found most enjoyable!”.

Jeremy further explored the challenges they faced, explaining that coordinating the non-technical pressures of working together remotely across three teams, often across time zones, during the pandemic put the teams’ communication and time management skills to the test. The teams managed to overcome this by hosting smaller, inter-team meetings regularly that tackled specific problems, allowing them to move through their strategy more effectively.

John McNamara, an IBM Master Inventor and the IBM UK University Programs Lead, was the coordinator managing the student IXN projects with IBM.

Jeremy Lo Ying Ping further speaks of how well the groups worked together and the support they received from UCL and IBM supervisors,

“We were lucky in that most of us were knew each other prior to starting our projects and that we all worked well together, both within and across our three teams. We were also fortunate to have received the full support of our main UCL and IBM supervisors Dr Dean Mohamedally, Dr Graham Roberts and Prof John McNamara, respectively.”

John McNamara explains the core motivations for the project, which aligns well with the UCL IXN ethos, to help those in need via technology;

“Everyone that chooses a career in technology has a number of motivations for doing so, however, if you scratch beneath the surface, they will all distil down to one prime ambition; and that is to make this world a better place for everyone. The motivation for this project was no different - to use creativity and ingenuity to build technology that can be used to make people's lives better. In this instance provide the means to make the benefit of technology (artificial intelligence) available to everyone in a consumable way. The IXN program is certainly a catalyst for breakthroughs. The program removes barriers to innovation by providing access to cutting edge technology, experienced mentorship, highly skilled students, and applies them to address industry and societal challenges. This is a wonderful recipe for breakthroughs, that allows us to ask the question "What if...?" and then go and build it.”
- Jon McNamara, IBM Master Inventor.

The successor v3.0 project teams will start in October 2021 with IBM and will be looking at developing new speech recognition features for wearables, smart TV integration and make use of proximity detection.

IBM FISE v2.0 students

• Jeremy Lo Ying Ping
• Akhere Ihoeghinlan
• Adam Piwowarczyk
• Mohammad Syed
• Calin Hadarean
• Daniel Javadinejad
• Radu Echim
• Ernest Nkansah-Badu

Editorial by Selina Peerbux

References:

IBM UK Newsroom - Announcements

UCL IXN's success sees further expansion via IBM collaboration 

IBM Works With UCL to Tackle Loneliness Through Digital Technologies