Orange is one of those words that famously have nothing perfectly to rhyme with them. The other one is silver. However, the Oxford Rhyming Dictionary does show both these words as having half-rhymes (such as lozenge with orange and salver with silver).
The principle of a half-rhyme in these cases is quite simple. Whereas a full and stressed rhyme (e.g. hand / stand) or even an unstressed rhyme (such as handing / standing) contain vowels that are common to both words, a half-rhyme like orange / lozenge or silver / salver (technically speaking pararhymes) has obvious differences between vowels in certain syllables.