It's been suggested that dairy in MS is harmful because of the saturated fat, and specific milk proteins mimicking a molecule that causes inflammation and demyelination. There's no conclusive evidence of this (ie. randomised controlled trials) but there are observational studies that show populations with higher dairy intake also have higher rates of MS (Nurses Health Study/China Study) and that it is associated with worsening disability in MS (HOLISM). Still not conclusive because these are people self-reporting diet, and can't be validated. But if avoiding dairy does have a beneficial effect, it's long-term decades-long phenomenon. It won't show after days/weeks/months. So I don't think anyone can really answer your question. I think you just have to look what evidence there is and decide for yourself.