Service evaluation of the impact of Shift.ms on the lives of its members

An Independent evaluation of Shift.ms by the Tavistock Institute

(Dr Dione Hills, Matthew Gieve, and David Drabble)

Research and evaluation carried out using 357 respondents between May–October 2013.

The evaluation looked for a range of impacts from using Shift.ms, relating to access to information, social and emotional support, health related behaviours, coping with MS and ‘getting on with life’.

Below are some of the highlights from the evaluation. To read the full executive summary, please click here


Overall Impacts

 

Impact on access to information

 

Impact on social and emotional support

 

Coping with MS and ‘getting on with life’

 

Breakdown of user base

 

Conclusions

 

 

Overall impacts

• Just under nine out of ten survey respondents reported taking some benefit from using Shift.ms

• Just over 80% of members reported feeling better informed

• Just over 70% of members reported a social or emotional benefit

• Just under 70% of members reported that using Shift.ms had had a beneficial impact on coping with their condition or getting on with their life

• Just under 60% of members reported a beneficial effect on their behaviour regarding their own health

Overall impacts_no line
back to top menu

 

Impact on access to information

As the table below shows, the most common types of information that members reported gaining are about other members’ experiences of having MS (72.1%) as well as tips and insights into coping with the condition (71.4%).

access to information

Regarding other members’ experiences, Interviewees spoke of learning about a wide range of issues including: diagnosis and disclosure; interactions with healthcare professionals; relationships with family and friends; and day to day coping with MS, as well as conversations that went beyond MS onto other aspect of the members’ lives.

“I have always managed to find what I am looking for. Whether I asked and they tell me where to find it or they told me, they always know where to find information:  “read this” or “speak to this person” or “go speak to your MS nurse” There is usually something there I can work with or I can ask somebody on the chat.” (Shift.ms member)


back to top menu

 

Impact on social and emotional support

The second key area for the evaluation was to look at whether using Shift.ms provides social and emotional support to its members. While only a small proportion of survey respondents said they specifically came for support (21%) over half of those surveyed said that they first joined the site to connect with other people with MS.

The most notable impact reported regarding emotional and social support is that over half (58%) of respondents agreed with the statement that they felt less isolated as a result of using Shift.ms. A smaller proportion (40%) agreed with the statement that they felt a sense of ownership and belonging to Shift.ms.

social and emotional support

 

This was mirrored by the interviews where several interviewees likened Shift.ms to ‘home’ or to ‘family’ indicating high levels of belonging and ownership.

“it is like going back home in a way, maybe I am exaggerating a bit, you feel supported just because there is a space where a lot of young people with MS are, you can share lots of stories, it is very supporting, like a home. Living with MS would be much more difficult in a way without Shift.ms.” (Shift.ms member)


back to top menu

Coping with MS and ‘getting on with life’

Ultimately the evaluation is interested in impacts on the members’ lives overall.  The graph below shows the percentage of respondents who agreed with the statement ‘Using Shift.ms has helped me deal with my condition in any of the following ways’.

getting on with life

The above chart shows 51 % of survey respondents agreed that Shift.ms had helped them to deal better with the uncertainties of having MS, while 42% agreed that their use of Shift.ms has made them feel less anxious about their MS.  Uncertainty and the corresponding anxiety were identified in the scoping phase as overarching features of diagnosis with MS that could have a big effect on people with MS’s wellbeing, particularly as many of those interviewed described anxiety and stress as having an exacerbating effect on symptoms.

 

“if my symptoms are bad one day then I will check Shift.ms more regularly, I have noticed that the more symptoms I have the more I go on, it feels like a relief to know there are people out there going through the same, this is even if I don’t post if I am just reading what other people are saying.” (Shift.ms member)


back to top menu

 

Breakdown of user base

To further analyse the impact of Shift.ms on its community, we looked at how different groups used the site, categorised by speed of joining after diagnosis; regularity of visits; and those who felt they had made friends through Shift.ms.

 

a) The impact by speed of joining after diagnosis

The research looked at those that had joined Shift.ms soon after diagnosis – defining quick joiners as those who joined within two years of diagnosis and slow joiners as those who took longer than two years to join after diagnosis.

We found that the group who were quicker to join Shift.ms reported greater benefits across the board, particularly with regard to social and emotional benefits.

 

b) The impact of regular use

A third of those surveyed reported using the site on at least a weekly basis (daily: 6%, a few times a week: 13%, weekly: 15%).  Taken as a group these members reported substantially higher benefits from using Shift.ms than those who used the site less frequently across the main categories.

We looked at the differences in response between those who use the site on at least a weekly basis and those who use it less regularly. Whilst the difference in any impact was statistically significant with 95% confidence, all other categories were significant at 99% confidence. This result is not altogether surprising as those that enjoy the service or find it more useful are more likely to use it.  But it is also true that the more you use the service, the more likely it is to impact on your life.

 

c) The impact of making friends

Just under a third of respondents (30%) reported having made friends with other members of Shift.ms. Of these, over half (55%) had had contact with people outside the website (through phone, email, or other social media platforms) and 20% had met in person. People who made friends were more likely to report that they liked the site’s tone, were regular visitors, were under 40 years old, have relapsing and remitting MS and were diagnosed less than 2 years ago.

Taken as a group, members that reported having made friends through Shift.ms reported higher benefits than others who did not report having made friends through Shift.ms.

back to top menu

Conclusions

The evaluation finds strong evidence that members of Shift.ms derive a wide range of different benefits from using the website. Overall almost nine out of ten survey respondents agreed that they had taken some benefit from using the site. In that sense there is something on the site for almost everyone. The survey shows that as a result of using Shift.ms the majority of members feel better informed, better supported, better able to take care of their own health, and better able to get on with their lives.  In addition a substantial minority (around 40%) of members felt that using Shift.ms had helped them come to terms with their condition.

While most members reported at least some benefits from using the site, within the overall population there is a smaller group of members that take a more substantial benefit. This is a group of regular users who tend to be both younger and more recently diagnosed. These groups systematically reported greater benefits from using the site.

“There are lots of young people and you can feel that, the way people communicate with one another, there is more energy than with other websites, lots of young people are joining the website.  One of the keys is the amount of young people who are involved in Shift and that really creates a different atmosphere.” (Shift.ms member)

The features of Shift.ms that lead to these impacts centre on the community and the emphasis placed on user-generated content. Indeed in some sense the language of ‘impacts’ does not fully capture the process that actually occurs on Shift.ms, as not only do members derive benefits from using it, they also create benefits for others. In that sense Shift.ms does not directly produce the impacts, but instead creates a space where members can interact, share information and support one another other.

“[Shift.ms] has done very well to give a sense of ownership to the community: “this is our space”. Although it has a strong identity it does not have as strong voice, it’s not just imposing its own ideas on people.” (Stakeholder)

The community is characterised by a sense of belonging and ownership by the members, and is seen as having welcoming and non-judgemental atmosphere. This sets the background for diverse interactions to take place, where members can share technical and medical information or provide more informal support and advice.

The factors that contribute to this atmosphere are the tone and appearance of the site, which encourages positive and respectful interaction and the comparative youth of the membership that is seen to bring a level of vibrancy to the site.

“It is amazing the amount of knowledge that the members have collectively, I don’t think there is a doctor in the world who has as much knowledge as the members” (Shift.ms member)

 

“I know there is a community out there and if I need them they would reassure me, and they can relate, and your family are nice but they don’t understand what you’re going through. It’s there if you need it, it is an emotional crutch.  The fact that I know it is there.” (Shift.ms member)

 

“The very fact of reading it even if you don’t respond sort of makes you feel that you’re not alone and there are people going through what you are going through” (Shift.ms member)

back to top menu

 

This independent evaluation was funded by Biogen Idec and Roche.