To make a menu item open in a modal popup, simply surround the menu title in {modal} tags.

So if your menu item title is Click here!, change it to {modal}Click here!{/modal}.

You can also pass extra variables in the tag as described earlier, like: {modal title="My Page"}Click here!{/modal} or {modal width="600" height="400"}Click here!{/modal}.

If for some reason you don't want to use the {modal} tags (or it doesn't work), you can also give the menu item a custom class name (works with most menu modules) and set Modals up to convert by that class name.