In last post, I talked about how to change log level and add diagnosis information per request at runtime. It helps us to debug and trouble shooting problems in production environment.
But at that time, we use X-DEBUG-KEY header, check whether it matches the key at server side, if so enable this feature.
This is unsafe(as we may return internal implementation information to client). - even we encrypt the X-DEBUG-KEY in the source code and change it before push to production.
Recently, we developed a common feature toggle service. So now we can use the feature toggle service to control who can use the debug request feature and turn it on/off dynamically at server side.
Feature Toggle is a quite simple but powerful service. With it, we can do A/B testing, we can test and trouble shooting on productions: such as we can allow some specific users to debug request, to mock user etc.
But at that time, we use X-DEBUG-KEY header, check whether it matches the key at server side, if so enable this feature.
This is unsafe(as we may return internal implementation information to client). - even we encrypt the X-DEBUG-KEY in the source code and change it before push to production.
Recently, we developed a common feature toggle service. So now we can use the feature toggle service to control who can use the debug request feature and turn it on/off dynamically at server side.
FeatureService
Admin can create/update/delete feature. The feature can be stored in database(sql or nosql).
Client will call isFeatureOn with feature name and a map. isFeatureOn will get the feature from db and check whether it's on. It will return Decision.NEUTRAL if not exist.
public interface FeatureService {
Decision isFeatureOn(String name, Map<String, Object> variables);
public Optional<Feature> getFeature(final String name);
public void addFeature(Feature feature);
public void updateFeature(Feature feature);
void deleteFeature(Iterator<String> iterator);
Collection<Feature> getAllFeatures();
}
Feature and Expression
Feature contains a list of expression, and will call the expression's apply method to check whether the feature should be on or off.
Currently we only support AnyMatchExpression, PercentageExpression and ScriptExpression, but it can be easily extended.
Notice we can use the optional ttlSeconds to control how long the feature will be on, after that it will be expired. This adds another safety layer to the feature toggle service.
-- We annotate updateTime with View.Viewable.class, so outside API can only view it but can't update it: check Using Jackson JSON View to Protect Mass Assignment Vulnerabilities
Currently we only support AnyMatchExpression, PercentageExpression and ScriptExpression, but it can be easily extended.
-- We annotate updateTime with View.Viewable.class, so outside API can only view it but can't update it: check Using Jackson JSON View to Protect Mass Assignment Vulnerabilities
public class Feature {
private String name;
private String appId;
/**
* This is used as extension, in case later we support rejectFirst.
*/
private boolean acceptFirst = true;
public Feature() {}
@JsonView(View.Editable.class)
private List<Expression> expressionList = new ArrayList<>();
@JsonView(View.Editable.class)
protected int ttlSeconds = -1;
@JsonView(View.Viewable.class) // all other properties are marked as Editable, except updateTime
private Date updateTime;
@JsonIgnore
public boolean isEffective() {
if (ttlSeconds != -1) {
if (updateTime == null) {
logger.error("Invalid feature {}, updateTime is null while timeToLive is {}", name, ttlSeconds);
throw new XXException(ErrorCode.INTERNAL_ERROR,
String.format("Invalid feature %s: ", getName()));
}
final MutableDateTime expiredTime = new MutableDateTime(updateTime);
expiredTime.addSeconds(ttlSeconds);
return expiredTime.isAfterNow();
}
return true;
}
public Decision isFeatureOn(final Map<String, String> variables) {
if (hasExpired()) {
return Decision.NEUTRAL;
}
final List<Expression> expressionList = getExpressionList();
for (final Expression expression : expressionList) {
if (expression.apply(variables).equals(Decision.ACCEPT)) {
return Decision.ACCEPT;
}
}
return Decision.NEUTRAL;
}
// ignore equals, hashcode, toString, getter, setter..
}
@JsonIgnoreProperties(ignoreUnknown = true)
@org.codehaus.jackson.annotate.JsonIgnoreProperties(ignoreUnknown = true)
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = AbstractExpression.FIELD_TYPE,
visible = false)
@JsonSubTypes({@Type(value = PercentageExpression.class, name = PercentageExpression.TYPE),
@Type(value = AnyMatchExpression.class, name = AnyMatchExpression.TYPE),
@Type(value = ScriptExpression.class, name = ScriptExpression.TYPE)})
public interface Expression {
public Decision apply(Map<String, Object> variables);
}
public class AnyMatchExpression extends AbstractExpression {
public static final String TYPE = "anyMatch";
/**
* which variable name to look up
*/
private String variableName;
private boolean caseInsensitive;
@JsonProperty("set")
private Set<String> set = new HashSet<>();
public AnyMatchExpression() {
type = TYPE;
}
@Override
public Decision apply(final Map<String, Object> variables) {
final Object object = variables.get(variableName);
if (object != null) {
final Iterable<String> idIt = SomeUtil.COMMA_SPLITTER.split(object.toString());
for (String id : idIt) {
if (caseInsensitive) {
id = id.toLowerCase();
}
if (set.contains(id)) {
return Decision.ACCEPT;
}
}
}
return Decision.NEUTRAL;
}
/**
* Precondition: There is no null value in the set. Otherwise it will throw NPE.
*/
public void setSet(final HashSet<String> set) {
if (caseInsensitive) {
final HashSet<String> lowercaseSet = new HashSet<>();
for (final String word : set) {
lowercaseSet.add(word.toLowerCase());
}
this.set = lowercaseSet;
} else {
this.set = set;
}
}
// ignore toString, getter, setter..
}
public class PercentageExpression extends AbstractExpression {
public static final String TYPE = "percentage";
/**
* a value between 0 and 100
*/
private int percentage;
public PercentageExpression() {
type = TYPE;
}
@Override
public Decision apply(final Map<String, Object> variables) {
if (percentage == 100) {
return Decision.ACCEPT;
}
final Random random = new Random();
final int rValue = random.nextInt(100);
if (rValue < percentage) {
return Decision.ACCEPT;
}
return Decision.REJECT;
}
}
Check Whether Debug Request Feature is On
Now we can check whether the debug request feature is turned on for this user. - we only do this when X_DEBUG_REQUEST header exists. - Of course we cache the Feature object, so we don't do db call every time.protected void checkDebugRequestFeature(final ContainerRequest request) {
if (debugRequestEnabled) {
final String debugRequest = request.getHeaderValue(X_DEBUG_REQUEST);
if (StringUtils.isNotBlank(debugRequest)) {
final User user = getUserObjectHere();
if (SomeUtil.isFeatureOn(SomeUtil.FEATURE_DEBUG_REQUEST, user, featureService)) {
MDC.put(X_DEBUG_REQUEST, debugRequest);
RequestContextUtil.getRequestContext().setEnableDebug(Boolean.valueOf(debugRequest));
} else {
RequestContextUtil.getRequestContext().setEnableDebug(false);
}
}
}
}
Other User Cases Using Feature ToggleFeature Toggle is a quite simple but powerful service. With it, we can do A/B testing, we can test and trouble shooting on productions: such as we can allow some specific users to debug request, to mock user etc.
