Many of the Stoics were pantheists. They thought the “all”, the universe, was the body of God. So everything was a part of God.
With Epictetus it is hard to tell. He talks about Zeus and the gods. He also talks about the “all”. Maybe he just used polytheism symbolically. His student, the emperor Marcus Aurelius, was a pantheist. But Epictetus himself is ambiguous.
I do not believe in either polytheism or pantheism, although polytheism seems to me a better fit for the pluralistic and multifaceted world that presents itself to us. So talking about gods makes some sense if you do not have a Judeo-Christian doctrine of creation. Epictetus, however, is not about ontology or metaphysics. His is a practical guide to living.
Suppose you are an atheist. Can you not benefit from the Bible’s Book of Proverbs? There is a lot there that appeals to common sense apart from religious doctrine. Just as an atheist might benefit from practical insights from ancient Jews, so people of any faith might benefit from the ancient wisdom of Epictetus.
Let me use another example. I have used books on Feng Shui, based on ancient Chinese religion, as a guide for decluttering some spaces. Some books on Feng Shui are mostly practical and do not do much with the worldview that underlies it. Others books, though, make it hard to separate the practice of Feng Shui from the forces and magic involved in the original philosophy. But Feng Shui does wonders for a room.
You have to assume that ancient spiritual traditions, whether Eastern, Pagan, Islamic, Amerind, or other, passed down some real wisdom along with some things we cannot accept. We impoverish ourselves if we reject everything that does not directly come from our own tradition.
In the self-help sections of most bookstores I see a lot or books based on humanistic psychology. In Christian bookstores, I see books based on humanistic psychology with “biblical principals” based on bad exegesis shoehorned in. I wonder if we wouldn’t be better off looking to some old traditions like Stoicism, Rabbinic Judaism, Catholic monasticism, and Confucianism that concerned themselves with practical life issues.