Mix filling. In a large bowl, combine the hot sauce, butter, dried oregano, celery and chicken. Divide into 6 equal parts.
Build quesadillas. Combine cheeses in a bowl and divide evenly into 12 parts. Brush one side of each tortilla lightly with olive oil. Place 6 tortillas oiled-side down on platter or wire rack. Layer each with one portion of the cheese blend, one portion of chicken, followed by another portion of cheese blend. Top with another tortilla, oiled side up.
Grill quesadillas. Heat grill to medium high. Place quesadillas directly on the grills (or in grill basket) and cook on one side until a peek underneath reveals charring or grill marks, then flip and cook on the other side until you see the same signs and the cheese is melted, about 6 minutes total. Serve warm with chilled dressing for dipping.