You'll have to create a button mapping for the video playlist window (should be "VideoPlayList") as well and eg map "blue" to "ContextMenu".
Otherwise the default (global) mapping is used, which is usually the PVR functions for color buttons.
If in doubt enable debug logging in kodi and check kodi log for "Window Init" messages - this will tell you which window is currently being used. The https://kodi.wiki/view/window_ids wiki page lists the window names (which you have to use in keymap xml files) and window xml files - eg if you see "MyPlaylist.xml" in kodi.log you'll have to use "videoplaylist" in keymap.
The kodi log will also show you the button press and the performed action.
so long,
Hias