The Problem
In previous post: Using ResultSpecification to Filter Annotator to Boost Opennlp UIMA Performance, I introduced how to use ResultSpecification to make OpenNLP.pear only run needed annotators.
But recently, we changed our content analzyer project to use UIMA-AS for better scale out. But UIMA-AS doesn't support specify ResultSpecification at client side, so we have to find other solutions.
Luckily UIMA provides a more common mechanism: feature structures to allow us to control annotator's behavioral characteristics.
Using Dedicated Feature Structure to Control Server Behavioral
This time, we will take RegExAnnotator.pear as example, as we have defined more than 10+ regex and entities in RegExAnnotator, and the client would specify which entities they are interested.
Client specify values of the feature: org.apache.uima.entities:entities, such as "ssn,creditcard,email", RegExAnnotator will check the setting and run only needed regex.
Specify Feature Value at Client Side
First we have one properties file uima.properties which define the mapping of entity name to the UIMA type:
regex_type_ssn=org.apache.uima.ssn
regex_type_CreditCard=org.apache.uima.CreditCardNumber
regex_type_Email=org.apache.uima.EmailAddress
RegExAnnotator will get the value of org.apache.uima.entities:entities, if it's set, then it will check all configured regex Concepts and add it to runConcepts if the concept produces one of the uima types of these entities.
UIMA References - feature structures
Apache UIMA Regular Expression Annotator Documentation
http://comments.gmane.org/gmane.comp.apache.uima.general/5866
In previous post: Using ResultSpecification to Filter Annotator to Boost Opennlp UIMA Performance, I introduced how to use ResultSpecification to make OpenNLP.pear only run needed annotators.
But recently, we changed our content analzyer project to use UIMA-AS for better scale out. But UIMA-AS doesn't support specify ResultSpecification at client side, so we have to find other solutions.
Luckily UIMA provides a more common mechanism: feature structures to allow us to control annotator's behavioral characteristics.
Using Dedicated Feature Structure to Control Server Behavioral
This time, we will take RegExAnnotator.pear as example, as we have defined more than 10+ regex and entities in RegExAnnotator, and the client would specify which entities they are interested.
Client specify values of the feature: org.apache.uima.entities:entities, such as "ssn,creditcard,email", RegExAnnotator will check the setting and run only needed regex.
Specify Feature Value at Client Side
First we have one properties file uima.properties which define the mapping of entity name to the UIMA type:
regex_type_ssn=org.apache.uima.ssn
regex_type_CreditCard=org.apache.uima.CreditCardNumber
regex_type_Email=org.apache.uima.EmailAddress
public class UIMAASService extends AbstractService {
private static final String UIMA_ENTITIES = "org.apache.uima.entities";
private static final String UIMA_ENTITIES_FS = UIMA_ENTITIES + ":entities";
private static Joiner joiner = Joiner.on(",");
public Result process(String text, String lang, List<String> types,
Long waitMillseconds) throws Exception {
CAS cas = this.ae.getCAS();
String casId;
try {
cas.setDocumentText(text);
cas.setDocumentLanguage(lang);
TypeSystem ts = cas.getTypeSystem();
Feature ft = ts.getFeatureByFullName(UIMA_ENTITIES_FS);
Type type = ts.getType(UIMA_ENTITIES);
if (type != null) {
// if remote annotator or pear supports type
// org.apache.uima.entities:entities, add it to indexes,
// otherwise do nothing.
FeatureStructure fs = cas.createFS(type);
fs.setStringValue(ft, joiner.join(types));
cas.addFsToIndexes(fs);
}
casId = this.ae.sendCAS(cas);
} catch (ResourceProcessException e) {
// http://t17251.apache-uima-general.apachetalk.us/uima-as-client-is-blocking-t17251.html
// The UIMA AS framework code throws an
// Exception and the application must catch it and release a CAS
// before continuing.
cas.release();
logger.error("Exception thrown when process cas " + cas, e);
throw e;
}
Result rst = this.listener.waitFinished(casId, waitMillseconds);
return rst;
}
protected static final Logger logger = LoggerFactory
.getLogger(UIMAASService.class);
private UimaAsynchronousEngine ae = null;
protected UimaAsListener listener;
private String serverUrl;
private String endpoint;
private static final int RETRIES = 10;
public UIMAASService(String serverUrl, String endpoint) {
this.serverUrl = serverUrl;
this.endpoint = endpoint;
}
public void configureAE() throws SimpleServerException, IOException,
XmlException, ResourceInitializationException {
boolean success = false;
for (int i = 0; (i < RETRIES) && (!success); i++) {
try {
this.ae = new BaseUIMAAsynchronousEngine_impl();
this.listener = new UimaAsListener(this);
this.ae.addStatusCallbackListener(this.listener);
Map<String, Object> deployCtx = new HashMap<String, Object>();
deployCtx.put("ServerURI", this.serverUrl);
deployCtx.put("Endpoint", this.endpoint);
deployCtx.put("Timeout", 60000);
deployCtx.put("CasPoolSize", 20);
deployCtx.put("GetMetaTimeout", 20000);
if (StringUtils.isNotBlank(System.getProperty("uimaDebug"))) {
deployCtx.put("-uimaEeDebug", Boolean.valueOf(true));
}
this.ae.initialize(deployCtx);
success = true;
} catch (ResourceInitializationException e) {
if (i < 10) {
logger.error(
getName()
+ " configureAE failed when deploy , will retry, retried times: "
+ i, e);
} else {
logger.error(getName()
+ " configureAE failed, retried times: " + i, e);
throw e;
}
}
}
configure(null);
}
public UimaAsListener getListener() {
return this.listener;
}
public void deployPear(String appHome, File pearFile, File installationDir,
String deployFileName) throws Exception {
PackageBrowser instPear = PackageInstaller.installPackage(
installationDir, pearFile, true);
File deployFile = new File(instPear.getRootDirectory(), deployFileName);
logger.info(getName() + " deployFile: " + deployFile);
updateDeployFile(deployFile, this.serverUrl, this.endpoint);
Map<String, Object> deployCtx = new HashMap<String, Object>();
deployCtx.put("DD2SpringXsltFilePath", new File(appHome,
"resources/uima/config/dd2spring.xsl").getAbsolutePath());
deployCtx.put(
"SaxonClasspath",
"file:"
+ new File(appHome, "resources/uima/lib/saxon8.jar")
.getAbsolutePath());
BaseUIMAAsynchronousEngine_impl tmpAE = new BaseUIMAAsynchronousEngine_impl();
tmpAE.deploy(deployFile.getAbsolutePath(), deployCtx);
logger.info(getName() + " deployed " + pearFile.getAbsolutePath());
}
private static void updateDeployFile(File deployFile, String serverUrl,
String endpoint) throws FileNotFoundException, IOException {
String fileContext = null;
InputStream is = new FileInputStream(deployFile);
try {
fileContext = IOUtils.toString(is);
} finally {
IOUtils.closeQuietly(is);
}
fileContext = fileContext.replace("${endpoint}", endpoint);
fileContext = fileContext.replace("${brokerURL}", serverUrl);
Object os = new FileOutputStream(deployFile);
try {
IOUtils.write(fileContext, (OutputStream) os);
} finally {
IOUtils.closeQuietly((OutputStream) os);
}
}
public void deployPear(File pearFile, File installationDir,
String deployFileName) throws Exception {
String appHome = System.getProperty("cv.app.running.home").trim();
deployPear(appHome, pearFile, installationDir, deployFileName);
}
}
Add Feature in RegExAnnotator.xml<typeSystemDescription>
<typeDescription>
<name>org.apache.uima.entities</name>
<description />
<supertypeName>uima.tcas.Annotation</supertypeName>
<features>
<featureDescription>
<name>entities</name>
<description/>
<rangeTypeName>uima.cas.String</rangeTypeName>
</featureDescription>
</features>
</typeDescription>
</typeSystemDescription>
Check Feature Value at RegExAnnotator
RegExAnnotator will get the value of org.apache.uima.entities:entities, if it's set, then it will check all configured regex Concepts and add it to runConcepts if the concept produces one of the uima types of these entities.
public class RegExAnnotator extends CasAnnotator_ImplBase {
private static final String UIMA_ENTITIES = "org.apache.uima.entities";
private static final String UIMA_ENTITIES_FS = UIMA_ENTITIES + ":entities";
public void process(CAS cas) throws AnalysisEngineProcessException {
TypeSystem ts = cas.getTypeSystem();
org.apache.uima.cas.Type entitiesType = ts.getType(UIMA_ENTITIES);
FSIterator<?> it = cas.getAnnotationIndex(entitiesType).iterator();
org.apache.uima.cas.Feature ft = ts
.getFeatureByFullName(UIMA_ENTITIES_FS);
String onlyRegexStr = null;
AnnotationFS entitiesFs = null;
while (it.hasNext()) {
// TODO this is kind of weird
AnnotationFS afs = (AnnotationFS) it.next();
if (afs.getStringValue(ft) != null) {
System.out.println(afs.getType().getName());
onlyRegexStr = afs.getStringValue(ft).trim();
entitiesFs = afs;
}
// onlyRegexStr = afs.getStringValue(ft).trim();
logger.log(Level.FINE, "Only run " + onlyRegexStr);
}
if (entitiesFs != null) {
cas.removeFsFromIndexes(entitiesFs);
}
List<String> types = null;
List<Concept> runConcepts = new ArrayList<Concept>();
if (onlyRegexStr != null) {
onlyRegexStr.split(",");
types = Arrays.asList(onlyRegexStr.split(","));
for (Concept concept : regexConcepts) {
Annotation[] annotations = concept.getAnnotations();
if (annotations != null) {
for (Annotation annotation : annotations) {
if (types.contains(annotation.getAnnotationType()
.getName())) {
runConcepts.add(concept);
break;
}
}
}
}
} else {
runConcepts = this.regexConcepts;
}
// change this.regexConcepts.length to local variable: runConcepts
for (int i = 0; i < runConcepts.size(); i++) {
// same and omitted...
}
}
}
ReferencesUIMA References - feature structures
Apache UIMA Regular Expression Annotator Documentation
http://comments.gmane.org/gmane.comp.apache.uima.general/5866
