Context of use findings (v0, current app)
When will they use the app in the real world?
What are their motivations, perceptions and concerns?
General
- FWMOMCare perceived as health and COVID-related first and foremost
- Usually ignore notifications from the app (e.g. salary survey)
- Report Health and Safe@Home are the most frequently used features. Medical Report used also, to a lesser extent.
- Usage of other features is non-existent
- When users are asked to find a function they are not familiar with, a common behaviour is to open the hamburger menu
Language & Translations
- MWs tend to use the app in English even if they have difficulty reading it
- Certain native languages (e.g. Tamil) use translations which are highly academic and difficult to understand.
- Many MWs tend to mix native language and English, especially when describing technical/UI microcopy terms (e.g. OTP, mobile number, profile)
- Consider engaging an interpreter to check if translations can be understood by low-literacy workers
Safe@Home
MWs indicated they do both Health Reporting and Safe@Home at the same time, twice a day (once in the morning and once in the evening)
- MWs perceive the address to be the most important info on the page to verify
Report Health
MWs said they usually take their temperature once in the morning and once in the evening
- Morning: If they are in a rush, they sometimes key in their temperature while on the commute to work (before 10am)
- Perception on that MOM will become aware if they miss keying in their temperature 3 times in a row
Reluctance to use Health Reporting function if they were feeling ill or had temperatures above 37.5 degrees:
- First action is to inform their employer or dorm operator instead and take instruction from them about whether they should complete their Health Report.
- In practice, some key in a lower temperature into FWMOMCare than what they actually measured. They said they were not able to do this when actually entering the worksite, as their employer re-took their temperature and publicly verified it on the spot, recording it in another app or log book.
Frustration due to requirement to use of multiple apps (FWMOMCare and employer apps) for recording temperature
MWs attach different levels of importance to different symptoms
- "Runny nose they will show up to work, but for cough and sneezing they will not show up"
- Belief that if they reported their symptoms through the app, that they would be isolated by MOM
Call Doctor
Few have used telemedicine services; their top concerns are:
- Who will pay
- Language barrier when communicating with doctor
- How they will receive medicine
Motivation:
- Obtain MCs, so they are paid even if they are unable to work
- Share doctor's diagnosis with employer so employer knows what to do
Relationship with employer influences usage:
- Some concerned that the Call Doctor costs money for their employer; they were scared that their work visas would not be renewed if they used the function without permission from employers
- Others pleasantly surprised to discover this feature, with MW describing that it was "easier than asking company HR to help me" and "taking less time than to go to hospital, no need to queue".
Medical Records
Pre-usage perception is that this feature contains:
- Non-COVID related information
- Results from his diabetes blood test
- "My medical health report"
- Lifestyle advice on how to cope with diabetes, based on his blood sugar levels
Some understanding of "Covid-19 Infection Status" and "Swab Test Appointment", but low familiarity Discharge Memo:
- Some believe that 'Discharge Memo' signifies he does not have COVID and can be discharged from hospital. They would check this section to confirm if they still had COVID
- Some believe that there is new appointment booking functionality suggested by "Swab Test Appointment"
- MWs not familiar with technical terms used in "Covid-19 Infection Status", such as "Serology Results", though they understand "Lab Results"
News & Stories
Pre-usage perception is that the feature contains:
- Information related to MOM or COVID ("how many people have virus")
- MOM reminders and posters
Post-usage, the topics important to them are:
- Work-related accidents and prevention [safety supervisor worker]
Conversations
Pre-usage perception is that this could be for:
- Money transfer
- Sharing his concerns and feedback
Post-usage, appropriate frequency for salary survey is perceived to be monthly.
One MW noted that SembCorp independently tracks salary non-payment:
- They conduct interviews with workers (employed by SembCorp subcontractors)
- If they find salary non-payment, they force the subcon to pay back workers
- During this process, the name of the worker who complained is not known to the subcontractor.
v0 (current app) usability issues
Issue
Feature(s) impacted
USER EMOTION: Observation
Outcome/Workaround
Type of issue (10 usability heuristics)
Users affected
Severity
Ease of fix
Registration
OTP
FRUSTRATION:
During registration, users try to key in their own preferred date format, and are unable to understand error message presented. Users do not notice the date format microcopy (YYYY-MM-DD).
During OTP reset, users are given a calendar picker instead, causing frustration for users as they scroll through many months to find their DoB.
ABANDON:
Unable to proceed without moderator prompting
Error prevention
P1
Critical
Easy
Report Health
GLOSSING OVER:
Some users may be able to guess some of the symptoms based on the icons (e.g. cough and running nose) but most symptoms are difficult to understand either through text or icons.
IGNORE FIELDS:
Users ignore these questions and focus on the temperature input instead, resulting in inaccurate health reports received from MWs
Consistency and standards
P1
P2
Major
Challenging
Report Health
CONCERN:
Users become concerned by the red message that pops up when a temperature, and sometimes mention that they believe they have inputted data "incorrectly".
DELAY:
After some delay, users usually continue to submit their health report
Error prevention
P4
Major
Average
Call Doctor
FRUSTRATION:
Users are usually not aware of that the app gives users the ability to speak to a doctor. Users don't notice the "Call-A-Doc" function on the bottom bar unless prompted by the moderator, and don't have an interest in checking it during normal exploration of the app.
USER WORKAROUNDS:
1) Re-do the Health Report flow to access the prompt to call a doctor. [User believes they cannot Call Doctor without completing the Health Report First]
2) Access the hamburger menu
Visibility of system status
P1
P2
P3
P4
Major
Challenging
Call Doctor
CONFUSION:
Upon encountering first screen of Call Doctor flow, some users believe that doctors will call them within 30 minutes, without them needing to take any further action.
ABANDON:
Without moderator prompting to continue, the user abandons the task.
Visibility of system status
P1
P2
Critical
Challenging
Call Doctor
FRUSTRATION:
Some users believe that they must actively call the doctor, instead of passively waiting for a call. Users also believe that they may be able to get medical advice from the doctor through messages, not just calls.
USER WORKAROUNDS:
1) Tapping on the top-righthand side phone icon on the first Call Doctor screen. User expectation is that they will be able to contact the doctor with this method, or obtain his number.
2) Tapping on the "Call-a-Doctor" H1 text on the first Call Doctor screen. User expectation is that they will be able to contact the doctor with this method, or obtain his number.
3) When being redirected to the IHIS page to video call a doctor, users notice the H1 blue text for "Video Call a Doctor for Foreign Workers" and try to click on it.
Visibility of system status
P1
P3
P4
Major
Challenging
Medical Records
FRUSTRATION:
To access Medical Records, some users opened the hamburger menu and clicked on "Report Health". Possible noun-verb confusion on "Medical Records" vs "Report Health".
ABANDON:
User gives up on accessing COVID test results through the app, and instead relies on hearing about results through other touchpoints (e.g. boss, or through SGWorkPass app).
Recognition rather than recall
P3
Major
Average
Medical Records
FRUSTRATION:
When prompted to find their COVID test results, some users first choose the "Swab Test Appointment" submenu, not "Covid-19 Infection Status"
ABANDON:
User gives up on accessing COVID test results through the app, and instead relies on hearing about results through other touchpoints (e.g. boss, or through SGWorkPass app).
Recognition rather than recall
P2
P4
Major
Easy
News & Stories
DELAY:
Users' first instinct is to link on the news story title text instead of the 'open' icon on the right side, causing a delay.
USER WORKAROUNDS:
Users usually click on the right side icon after a few unsuccessful attempts of clicking on the title text.
Consistency and standards
P2
Minor
Easy
Context of use findings (v1)
v1 usability issues
Issue
Feature(s) impacted
USER EMOTION: Observation
Outcome/Expected Functionality
Type of issue (10 usability heuristics)
Users affected
Severity
Ease of fix
Registration
FRUSTRATION:
Users not living in dorms find that the address input is not suitable, especially the "floor number" for workers living in a private house.
INCORRECT ENTRY:
Users will enter the wrong data into the "floor number", usually the street number instead.
SUGGEST:
Indicate that floor number is an optional field
User control and freedom
P7
Minor
Easy
Registration
FRUSTRATION:
Users find that the DoB input of YYYY/MM/DD is not something they normally use. Their common usage is DD/MM/YYYY.
DELAY:
Most users are still able to enter the DoB field normally
Consistency and standards
P7
Cosmetic
Easy
Login
FRUSTRATION:
Users do not perceive that their date of birth is being used as a form of identity verification during login, and see this as repeated information from registration.
DELAY:
User continues to enter login, but is frustrated by need to repeatedly confirm date of birth every time they log in.
Flexibility and efficiency of use
P5
Minor
Average
OTP
Login
FRUSTRATION:
Workers struggle when switching between FWMOMCare and Messages app to obtain the OTP number
FRUSTRATION:
Notification for OTP SMS disappears quickly, before users are able to realize it
SUGGEST:
1) Extending the validity time of OTP number (e.g. valid for 5 min instead of just 30 seconds)
2) Provide instructions or shortcut guiding user on recognising the OTP SMS, and how to access messages function
3) Ask for permission to auto-fill OTP based on incoming SMS
4) Do away with OTP requirement and rely on PIN or just DoB
Visibility of system status
P7
Major
Challenging
Call Doctor
Survey
Report Health
News & Stories
CONFUSION:
Most users were confused about the purpose of this element.
EXPECTED FUNCTIONALITY:
Users thought this element was to:
1) Watch a video [especially if element is paired with a static image, e.g. at the end of Report Health flow or in the News flow]
2) Communicate directly with the doctor immediately [especially the element at the end of the Report Health flow]
3) Send a video/voice message to the doctor [e.g. first or last screen of Call Doctor flow, where the element is placed close to WhatsApp logo] - "If my condition very urgent and cannot wait, I will send voice message to doctor and say 'Sir I cannot wait, can you come earlier'. But maybe if critical I will just call 995, no need wasting time using app"
Consistency and standards
P5
P9
Minor
Challenging
Report Health
CONFUSION:
Users do not perceive the "+" icon as leading to starting the Report Health flow.
USER WORKAROUND:
Click on the microcopy "Morning" or the empty check box next to it instead to begin the flow, or Medical, or Safe@Home, or Call Doctor ("if your temperature is high, MOM says you need to speak to a doctor").
SUGGESTION:
Allow user to begin the flow by clicking on the checkbox, or on the microcopy "Morning"
Consistency and standards
P7
P8
Major
Easy
Report Health
CONFUSION:
If they enter that they are "Unwell" in Report Health, users expect that a doctor will contact them regardless of whether they answered yes or no to the final question "Would you like to see a doctor?"
EXPECTED FUNCTIONALITY:
Perception is that answering "Yes" means that the user will speak to a doctor immediately [high priority, urgent illness], while answering "No" means that the doctor will call the user later on [low priority, less urgent illness].
User control and freedom
P5
Minor
Average
Call Doctor
CONFUSION:
"Call Doctor" name of the feature leads people to believe that it allows the user to actively call a doctor. Users are confused when they read the first screen of Call Doctor, which suggests instead that a doctor will call them.
Other users believe that this function is for wayfinding or booking of appointments in hospitals: "I think this will lead me to the nearest clinic to visit a doctor....or to book an appointment to see a doctor" [P8, SPass Marine]
EXPECTED FUNCTIONALITY:
Feature allows for users to actively call the doctor. Perception that the recording below Step 3 of the first screen of Call Doctor may contain the doctor's phone number.
SUGGESTION:
1) Re-word name of feature, or use a different icon to indicate that the call is not immediate (e.g. Delayed Call icon).
2) Explain why users are not able to call the doctor immediately (e.g. service limitations, 'all doctors are currently engaged, but will call you within x hours')
3) Provide advice for users on how to proceed if their medical need is urgent (e.g. call 995)
Flexibility and efficiency of use
P7
P8
Minor
Average
Call Doctor
CONCERN:
On the first screen of Call Doctor, even for users who understand that it is a passive function where the doctor will call them, they have the following concerns (1 and 2 being the main concerns):
1) "How will I get the medicine?"
2) "How long will it take for a doctor to call me?"
3) "What type of doctor will call me?"
4) "Where is the doctor located?"
SUGGESTION:
Surface this information in a submenu on the first page
Help and documentation
P8
Minor
Easy
Call Doctor
OTP
CONFUSION:
Some users may have two phones; one to receive OTPs and make landline calls; another for data usage and WhatsApp. Users may have a preference for a landline call instead of WhatsApp. Users are not sue on how to express this preference.
"Some workers may also turn off their data network to save battery life...what if the doctor tries to call this number when the worker has turned off their data?" - P9
EXPECTED FUNCTIONALITY:
When asked "I would like a doctor to contact me via whatsapp", users expect to see an option for doctor to contact them via landline instead.
SUGGESTION:
1) Allow users to enter in an alternate number for landline calls
2) Ask users to confirm that they use WhatsApp for the number listed on the "Step 1" screen
3) Instruct users to ensure that their data is turned on for the window of time specified on the last screen
User control and freedom
P7
P9
Major
Challenging
Call Doctor
CONFUSION:
Perception is that if users report that they have a high temperature or have symptoms in the reporting flow, that doctors will call them regardless of how they answer the question in Step 2: "I would like a doctor to contact me via whatsapp"
→ This perception is stronger among CMP industries than other industries (may be because of MOM's stronger on-the-ground engagement with workers in CMP)
Step 3 of flow: Users also perceive that the recording at the end of the flow will include information about how to deal with their symptoms in the short-term; "some tips on what I can do, general info on my sickness".
EXPECTED FUNCTIONALITY:
Step 2: "I would like a doctor to contact me via whatsapp" - question influences how quickly a doctor will call them ('No' is less urgent); doctor will still call them regardless.
SUGGESTION:
1) Move the question "I would like a doctor to contact me via WhatsApp" from Step 2 to Step 1, and place it together with the phone number confirmation
2) Create a video which tells users what to do while waiting for a doctor to call them, customized for the symptoms they have inputted. E.g. if user lists that they are sneezing or having a running dose, show a video from a native language speaking doctor telling them to sit away from other workers and to inform their employer that they are awaiting a call from a doctor.
3) Provide advice for users on how to proceed if their medical need is urgent (e.g. call 995)
User control and freedom
P7
Minor
Challenging
Medical
CONFUSION:
"In Covid-19 Infection Status", some users interpret the H2 header "As at 17-12-2020" to mean that they have not yet received their latest COVID test results. Users believe that the results of the COVID tests would be underneath the other H2 headers "Lab Result(s)" and "Serology Results(s)", rather than the date listed at the top.
Also, most users are unable to understand what "Serology Results" mean.
ABANDON:
Users believe that they would need to wait to receive their COVID test results.
Visibility of system status
P5
Major
Challenging
News & Stories
CONFUSION:
Users usually do not click View More when viewing stories. If they interact with the story, they usually click on the title or the photo.
Most users do not expect to see additional information in text (e.g. on the mental health helpline number) about the News item when clicking 'View More'
EXPECTED FUNCTIONALITY:
E.g. "I thought View More means I can see some more videos or audio [not text about the video]"
SUGGESTION:
A) Use 'Details' / 'Further Information' in the microcopy instead of 'View More'
B) Show preview of words above microcopy to indicate that users will see more content if they click on this element
C) Expand story to new page if they click on title or image.
Consistency and standards
P5
P7
P8
Minor
Survey
CONFUSION:
Users' initial impression of "Survey" is that they can obtain information from the function, in addition to the user sharing information with MOM.
Users have a 'topic-based' mental model, rather than action-based.
EXPECTED FUNCTIONALITY:
E.g. Looking at the Survey icon on the home screen: "It will be good if it tells me which medical [service] I should go if I have shoulder pain, back pain and how much it costs. Online [health] providers should be listed here in the Survey. It will be able to tell me if my doctor has changed to a different hospital. Different hospitals will share information to this feature" [P5]
"Mental health helpline...it should be in Survey....people are expecting us to talk something" [P7]
SUGGESTION:
A) At the end of each survey, surface info link to relevant resources to obtain more help on that particular issue (e.g. FAQ or helpline)
B) Communicate clearly what MWs get out of completing a survey
Match between system and real world
P5
P7
Minor
Survey
CONCERN:
A) Timeliness of response to salary survey: "If I report [some issue by] today, can they attend to my complaint by today?"
→ "They will call me back, or use the app. I prefer a call, so I can talk to the person more about the issue"
B) "How will they solve my problem without contacting my employer? I don't believe it."
SUGGESTION:
At the end of unhappy salary survey flow, state a timeframe about how long it will take for MOM to call them to respond to the complaint.
Visibility of system status
P8
Minor
Context of use findings (v2)
When will they use the app in the real world?
What are their motivations, perceptions and concerns?
Registration
Verification
- FWMC perceived to be actively verifiying name, DoB, FIN, mobile number and address input as part of registration process. Perception that any incorrect entry by MW would prevent successful registration
- Some MWs were frustrated about the number of 'verifications' required by MOM before registration, but accepted the need for MOM to ask the information
- Date of Birth: One MW (Indonesian MDW) said that she would not give her actual date of birth for cultural reasons - "if people know your DOB they can either charm or curse you, so usually I will not give the completely correct date. E.g. gmail, date, month is correct, but year is different"
- Address: One MW (Filipino service worker) perceived that if he updated his address in FWMC or other government apps, that his employer would be automatically notified of the address change as well. He does not update his employer about changes of address until he needs to renew his work pass.
One-time PIN (OTP)
- Most MWs interviewed had not experienced automated entry OTP from SMS.
- All MWs interviewed comfortable with FWMC asking for phone permission to read SMSs for the purposes of smoother OTP entry, "as long as they don’t take any personal info [for advertising]" because MOM is a "trusted agency".
Language choice
- Most MWs said use the English language version despite their language being offered.
- Most MWs are also not aware that they can choose a different language after their initial choice during registration
- Concern among newer MWs:
- If they select their native language, perception that this would change their keyboard settings and they would be forced to type in another keyboard (most perceived difficulty in operating or switching between different keyboards)
- Practices:
- All MWs interviewed except Chinese use English as a system language for their phone, not their native language.
- Most MWs type messages in English/English transliteration of native language and also rely heavily on voice memos
Report Health
Frequency of use
- MWs said that using FWMC to report their temperature daily was less important due to gradual relaxation of COVID measures [pre-Phase 2 Heightened Alert]
- One MW (Indian construction worker) said that it was more important to let their supervisor know if they had a high temperature, rather than use FWMC to report their temperature. Another MW said that they would only put in their temperature in FWMC after receiving medical treatment, if at all
Video Call a Doctor
Pre-usage perceptions
- Residential MWs not in CMP industries saw that this function was for emergency use, or to call for medical assistance for someone else (similar to 995). One MDW said she could not imagine how a "checkup can be done on the phone...it's impossible...I cannot imagine".
- Preference among residential MWs to find their own medical care instead of using this app. E.g. "why do you need to use a government app to call a doctor? General practitioner is fast, you will engage with the reception people, they can let you see the doctor. If it’s fully packed, then they will give an appointment."
- For dorm-based MWs, mixed reaction. Some had no idea what it was for and expressed hesitation about using this function (especially without employer approval), while others understood the purpose of video calling a doctor from the home page alone and said they would use it if their employer did not provide medical assistance.
Post-usage perceptions
- Even when assured that the video call is free, dorm and CMP MWs strongly expressed a preference to get medical assistance through their employer or with assistance from doctors stationed in their dormitories, especially during working hours.
- Dorm and CMP MWs said they would mainly use this function after working hours for non-urgent medical issues (e.g. "headache"), and usually only after checking with their employer.
- For serious ("heart pain and stomach pain") and emergency cases, dorm and CMP MWs would ask the dorm operator to arrange for an ambulance and go to the hospital as a priority. Informing the employer is a secondary priority in these cases.
- After realizing that it may take 2 hours for a doctor to come back to them, MWs said that they would only use this for non-urgent and non-serious medical issues; "2 hours is too long [if it's serious], 30 mins is okay"
Survey
As detailed in the usability issues below, most users did not associate "Survey" with sharing information with MOM on issues in Singapore, nor with getting help for late salary payments. Users had to be prompted by the moderator to explore this feature.
- [After being prompted to explore the feature] Most MWs said that this feature was useful, especially the ability to get help from MOM without their employers being informed unless they consented. "They promise me they won’t call the employer, this one is very important. Worker level they won’t be confident, so this message shows them the confidence." (Indian construction supervisor)
- MWs did not understand the regular information collection aspect of this feature, only the ability to get help for late salary payments. Accordingly, most MWs said they would only use this feature after not being paid for at least two months, out of goodwill to their employers. “I can one-two month waiting without salary, it’s ok. [After] three months, then I go MOM complain, [that's] better” (Indian construction worker)
- Even after knowing that they could get help for salary issues through this feature, some MWs said that they preferred other channels for this issue. "If I call them and explain to them in detail, but if I use this survey I cannot explain in detail" (Bangla construction worker)
News
Pre-usage perception
- Residential MWs saw this as being a general news feature, about government regulations and important updates on COVID
- Most dorm and CMP MWs, based on their perceptions of FWMC as a health-focused app, assessed that it contained health and Covid-related news only. Only one MW assessed that the feature would contain other topics (e.g. workplace safety).
- Most MWs did not consider topics like "mental health helplines" to be part of "health and Covid-related news", and did not expect the News feature to have this information.
- Most said they would find more information about a "mental health helpline" from MOM's posters or by getting emotional support from a friend/workplace.
Medical
Pre-usage perception
- Most CMP and dorm MWs were already familiar with Medical label
- Residential MWs expected to be able to contact a doctor or learn about medical benefits (e.g. free medical check-ups provided by the government) through this feature
- Some MWs are concerned that if personal medical data is accessible or stored through the app, that employers may have access to it as well. "Workers' mindset if that employers know they are not healthy, that they will be sent home" (Indian construction officer)