Preheat the oven to 350° F. Cream together butter and sugar in a stand mixer or electric hand mixer. Add eggs one at a time.
Add vanilla and cocoa powder; mix together until combined. Add flour one cup at a time and mix until combined.
Pour batter in a greased 9x13 pan and bake 30-35 minutes. Spread on miniature marshmallows and put back in the oven just long enough for the marshmallows to melt (about 3 minutes). Allow to cool completely before frosting.
Icing
Heat butter, milk, and cocoa powder in a saucepan to a boil. Add the powdered sugar, starting with one cup, and add more till desired sweetness is reached.
Pour over cooled brownies. Allow to cool before cutting and serving.