Place first 5 ingredients on a double thickness of cheesecloth. Gather corners of cloth to enclose seasonings; tie securely with string. Place spice bag and water in a 5- or 6-qt. slow cooker. Cook, covered, on low 8 hours. Discard spice bag.
Add tea bags; steep, covered, 3-5 minutes according to taste. Discard tea bags. Stir in milk; heat through. Serve warm.