Make the spicy mayo: In a small bowl, whisk together the mayonnaise, lemon juice and zest, Old Bay seasoning and cayenne. Season with salt.
Pour enough oil into a 3-quart heavy-bottomed saucepan until it reaches a depth of 2 inches. Heat the oil until a deep-fry thermometer reads 350°.
In a food processor, pulse the panko until finely ground.
Place the panko, flour and egg into 3 separate bowls. Stir the baking powder into the flour.
Working with one pickle at a time, dredge in flour, allowing any excess to fall. Dip each pickle into the egg then coat evenly in panko bread crumbs.
Working in batches, fry the pickles, turning halfway through, until crisp and light golden brown, 3 to 4 minutes. Using a slotted spoon, remove the pickles and transfer to a paper-towel lined plate and season immediately with salt. Serve with the spicy mayo.