public boolean init(StepMetaInterface smi, StepDataInterface sdi) { meta = (StreamLookupMeta) smi; data = (StreamLookupData) sdi; if (super.init(smi, sdi)) { data.readLookupValues = true; return true; } return false; }
public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException { meta = (StreamLookupMeta) smi; data = (StreamLookupData) sdi; if (data.readLookupValues) { data.readLookupValues = false; if (!readLookupValues()) // Read values in lookup table (look) { logError( BaseMessages.getString( PKG, "StreamLookup.Log.UnableToReadDataFromLookupStream")); // $NON-NLS-1$ setErrors(1); stopAll(); return false; } // At this point, all the values in the cache are of normal storage data type... // We should reflect this in the metadata... // if (data.keyMeta != null) { // null when no rows coming from lookup stream for (ValueMetaInterface valueMeta : data.keyMeta.getValueMetaList()) { valueMeta.setStorageType(ValueMetaInterface.STORAGE_TYPE_NORMAL); } } if (data.valueMeta != null) { // null when no rows coming from lookup stream for (ValueMetaInterface valueMeta : data.valueMeta.getValueMetaList()) { valueMeta.setStorageType(ValueMetaInterface.STORAGE_TYPE_NORMAL); } } return true; } Object[] r = getRow(); // Get row from input rowset & set row busy! if (r == null) // no more input to be expected... { if (log.isDetailed()) logDetailed( BaseMessages.getString( PKG, "StreamLookup.Log.StoppedProcessingWithEmpty", getLinesRead() + "")); // $NON-NLS-1$ //$NON-NLS-2$ setOutputDone(); return false; } if (first) { first = false; // read the lookup values! data.keynrs = new int[meta.getKeystream().length]; data.lookupMeta = new RowMeta(); data.convertKeysToNative = new boolean[meta.getKeystream().length]; for (int i = 0; i < meta.getKeystream().length; i++) { // Find the keynr in the row (only once) data.keynrs[i] = getInputRowMeta().indexOfValue(meta.getKeystream()[i]); if (data.keynrs[i] < 0) { throw new KettleStepException( BaseMessages.getString( PKG, "StreamLookup.Log.FieldNotFound", meta.getKeystream()[i], "" + getInputRowMeta().getString(r))); // $NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ } else { if (log.isDetailed()) logDetailed( BaseMessages.getString( PKG, "StreamLookup.Log.FieldInfo", meta.getKeystream()[i], "" + data.keynrs[i])); // $NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ } data.lookupMeta.addValueMeta(getInputRowMeta().getValueMeta(data.keynrs[i]).clone()); // If we have binary storage data coming in, we convert it to normal data storage. // The storage in the lookup data store is also normal data storage. TODO: enforce normal // data storage?? // data.convertKeysToNative[i] = getInputRowMeta().getValueMeta(data.keynrs[i]).isStorageBinaryString(); } data.outputRowMeta = getInputRowMeta().clone(); meta.getFields( data.outputRowMeta, getStepname(), new RowMetaInterface[] {data.infoMeta}, null, this); // Handle the NULL values (not found...) handleNullIf(); } Object[] outputRow = lookupValues(getInputRowMeta(), r); // Do the actual lookup in the hastable. if (outputRow == null) { setOutputDone(); // signal end to receiver(s) return false; } putRow(data.outputRowMeta, outputRow); // copy row to output rowset(s); if (checkFeedback(getLinesRead())) { if (log.isBasic()) logBasic( BaseMessages.getString(PKG, "StreamLookup.Log.LineNumber") + getLinesRead()); //$NON-NLS-1$ } return true; }