Joshua
open source statistical hierarchical phrase-based machine translation system
|
Public Member Functions | |
Cell (Chart chart, int goalSymID) | |
Cell (Chart chart, int goal_sym_id, int constraint_symbol_id) | |
Set< Integer > | getKeySet () |
SuperNode | getSuperNode (int lhs) |
Package Functions | |
boolean | transitToGoal (Cell bin, List< FeatureFunction > featureFunctions, int sentenceLength) |
HGNode | addHyperEdgeInCell (ComputeNodeResult result, Rule rule, int i, int j, List< HGNode > ants, SourcePath srcPath, boolean noPrune) |
List< HGNode > | getSortedNodes () |
Map< Integer, SuperNode > | getSortedSuperItems () |
Private Member Functions | |
void | addNewNode (HGNode node) |
void | ensureSorted () |
Private Attributes | |
Chart | chart = null |
int | goalSymbol |
HashMap< HGNode.Signature, HGNode > | nodesSigTbl = new HashMap<HGNode.Signature, HGNode>() |
Map< Integer, SuperNode > | superNodesTbl = new HashMap<Integer, SuperNode>() |
List< HGNode > | sortedNodes = null |
Static Private Attributes | |
static final Logger | logger = Logger.getLogger(Cell.class.getName()) |
this class implement functions: (1) combine small itesm into larger ones using rules, and create items and hyper-edges to construct a hyper-graph, (2) evaluate model score for items, (3) cube-pruning Note: Bin creates Items, but not all Items will be used in the hyper-graph
joshua.decoder.chart_parser.Cell.Cell | ( | Chart | chart, |
int | goalSymID | ||
) |
joshua.decoder.chart_parser.Cell.Cell | ( | Chart | chart, |
int | goal_sym_id, | ||
int | constraint_symbol_id | ||
) |
HGNode joshua.decoder.chart_parser.Cell.addHyperEdgeInCell | ( | ComputeNodeResult | result, |
Rule | rule, | ||
int | i, | ||
int | j, | ||
List< HGNode > | ants, | ||
SourcePath | srcPath, | ||
boolean | noPrune | ||
) | [package] |
a note about pruning: when a hyperedge gets created, it first needs to pass through shouldPruneEdge filter. Then, if it does not trigger a new node (i.e. will be merged to an old node), then does not trigger pruningNodes. If it does trigger a new node (either because its signature is new or because its logP is better than the old node's logP), then it will trigger pruningNodes, which might causes other nodes got pruned as well Creates a new hyperedge and adds it to the chart, subject to pruning. The logic of this function is as follows: if the pruner permits the edge to be added, we build the new edge, which ends in an HGNode. If this is the first time we've built an HGNode for this point in the graph, it gets added automatically. Otherwise, we add the hyperedge to the existing HGNode, possibly updating the HGNode's cache of the best incoming hyperedge.
Here, the edge has passed pre-pruning. The edge will be added to the chart in one of three ways:
each node has a list of hyperedges, need to check whether the node is already exist, if yes, just add the hyperedges, this may change the best logP of the node
the position of oldItem in this.heapItems may change, basically, we should remove the oldItem, and re-insert it (linear time), this is too expense)
void joshua.decoder.chart_parser.Cell.addNewNode | ( | HGNode | node | ) | [private] |
two cases this function gets called (1) a new hyperedge leads to a non-existing node signature (2) a new hyperedge's signature matches an old node's signature, but the best-logp of old node is worse than the new hyperedge's logP
void joshua.decoder.chart_parser.Cell.ensureSorted | ( | ) | [private] |
get a sorted list of Nodes in the cell, and also make sure the list of node in any SuperItem is sorted, this will be called only necessary, which means that the list is not always sorted, mainly needed for goal_bin and cube-pruning
sort the node in an decreasing-LogP order
Set<Integer> joshua.decoder.chart_parser.Cell.getKeySet | ( | ) |
List<HGNode> joshua.decoder.chart_parser.Cell.getSortedNodes | ( | ) | [package] |
Map<Integer, SuperNode> joshua.decoder.chart_parser.Cell.getSortedSuperItems | ( | ) | [package] |
boolean joshua.decoder.chart_parser.Cell.transitToGoal | ( | Cell | bin, |
List< FeatureFunction > | featureFunctions, | ||
int | sentenceLength | ||
) | [package] |
This function loops over all items in the top-level bin (covering the input sentence from <s> ... </s>), looking for items with the goal LHS. For each of these, add all the items with GOAL_SYM state into the goal bin the goal bin has only one Item, which itself has many hyperedges only "goal bin" should call this function
Chart joshua.decoder.chart_parser.Cell.chart = null [private] |
int joshua.decoder.chart_parser.Cell.goalSymbol [private] |
final Logger joshua.decoder.chart_parser.Cell.logger = Logger.getLogger(Cell.class.getName()) [static, private] |
HashMap<HGNode.Signature, HGNode> joshua.decoder.chart_parser.Cell.nodesSigTbl = new HashMap<HGNode.Signature, HGNode>() [private] |
List<HGNode> joshua.decoder.chart_parser.Cell.sortedNodes = null [private] |
sort values in nodesSigTbl, we need this list when necessary
Map<Integer, SuperNode> joshua.decoder.chart_parser.Cell.superNodesTbl = new HashMap<Integer, SuperNode>() [private] |