Nope, it’s fine. You’ll get more roastiness and bitterness from mashing it, but it won’t ruin the batch. It’ll still be a good dry Irish stout, even if it has more roast than Guinness does.
I sometimes add the roasted barley right at the end of the mash, with like 10 minutes remaining. It contributes mostly color at that point. But I’ve done it both ways, and it’ll turn out fine either way.
Edit - flaked barley should be mashed. Unless you want to leave starch in your beer, which you don’t with a DIS.