According to Papworth and Burke, the Arms of the old Augustinian Priory of our Lady at Walsingham were Argent on a cross sable five billets of the first: i.e. a silver shield with a black cross and on the black cross five silver rectangles. In use at the newly restored Shrine for donkey's years - I suspect Fr Fynes Clinton may have had a hand in all this; Heraldry was the sort of thing he was into - were the arms Silver; black cross; on the cross five lilies ... or so I think; I am going by memory. Also, in the first quarter was a golden representation of the Holy House: a breach of the usual convention that something gold cannot heraldically sit on something silver. However, this convention was very occasionally breached for overwhelmingly compelling reasons - such as in the Arms of Jerusalem. My theory is that HP or FC thought that the dignity of our Lady's shrine provided another overwhelmingly compelling reason! Incidentally, I have no knowledge whether the College of Heralds granted these arms; my hunch is that they were adopted without anybody troubling Queen Victoria Street.
None of us is supposed to use the arms of somebody else (not even if they have the same surname as ourselves!). If you have genuine connexion with some person, place or institution you may have arms which are like theirs but with some change to show the 'difference'. Thus the Holy See granted the see of Westminster arms indistinguishable from those of Canterbury except that they were 'differenced' by having the shield red rather than blue. The Holy See does not, I think, nowadays grant arms; although it does apparently expect bishops and those who in law are equivalent to bishops to have arms. The English heralds do still grant arms, but there are said to be legal reasons why they cannot grant arms to RC dioceses.
If anyone were ever to want arms - I can't think why they would - which were related to those of the Priory and/or shrine at Walsingham, the silver shield with a black cross and the lilies on the cross would be a good start; you could then 'difference' it by changing the colour of the cross.