Heat a large stockpot over medium heat, add oil and butter, then onion and carrots. Cook for 3-4 minutes until softened. Add garlic and rosemary, cook for 30 seconds.
Add chicken broth and salt, bring to a boil. Cook for 10 minutes, add orzo, and cook for another 8-10 minutes until tender. Remove from heat.
Whisk eggs, yolks, and lemon juice in a heatproof container until combined.
Slowly drizzle hot soup into egg mixture while whisking until about 2 cups are added.
Return pot to heat, add egg mixture, and cook over medium-low for 5 minutes, stirring until thickened. Add chicken, dill, and lemon zest. Adjust broth, salt, and lemon juice as needed. Serve garnished with lemon slices and dill.
Notes
For a creamier soup, add more eggs or a splash of cream.