Not all languages have alphabets: some, such as Chinese, use logograms (characters) and others use syllables as their basic units—for example Japanese (in one of its systems). But for the languages that do have alphabets, in the modern age dictionaries tend to be in alphabetical order—partly because that’s what we mean by a dictionary. If it were arranged by subject, we’d probably call it a thesaurus.
In seventeenth-century English dictionaries things were more haphazard with words alphabetized by first letter only. This can be found into the twentieth century in some glossaries. But for most dictionaries we can safely assume the answer is yes.
