See the attached graphic...it is from the wiki. The item circled in red is the firmware that you use to flash your etherdongle to output DMX. The one circled in blue is the one you drop in your /plugin/output folder as Rick stated. It should then show up in your list of available output plugins. Highlight it and select Use. It should move to the right side. I am not sure how to set it up past there as I have not used the ED in this mode, but it looks like there is no configuration other than channels...which would be all of them just like pixelnet.
Something I noticed is it appears that both plugins (pixelnet & DMX) have the same name in the plugin list when you go to select it in Vixen so I would delete the existing one from your output folder so that when you put the DMX one in there, you know that you are selecting the correct one; otherwise, I am not sure how you would know that you picked the right one to use.
-Keith