/** * 判断两个表所在节点集合是否相等 * * @param curRNodeSet 当前表所在的节点集合 * @param newNodeSet 新表所在节点集合 * @return 返回fase(不相等)或true(相等) * @author mycat */ private static boolean checkIfValidMultiTableRoute( Set<String> curRNodeSet, Collection<String> newNodeSet) { if (curRNodeSet.size() != newNodeSet.size()) { return false; } else { for (String dataNode : newNodeSet) { if (!curRNodeSet.contains(dataNode)) { return false; } } } return true; }
private RouteResultset routeWithPartitionColumn( SchemaConfig schema, TableConfig tc, QueryTreeNode ast, InsertParseInf parsInf, RouteResultset rrs, String stmt, LayerCachePool cachePool) throws SQLNonTransientException { String partColumn = tc.getPartitionColumn(); if (partColumn != null) { String sharindVal = parsInf.columnPairMap.get(partColumn); if (sharindVal != null) { Set<ColumnRoutePair> col2Val = new HashSet<ColumnRoutePair>(1); col2Val.add(new ColumnRoutePair(sharindVal)); return tryRouteForTable(ast, schema, rrs, false, stmt, tc, col2Val, null, cachePool); } else { // must provide sharding_id when insert String inf = "bad insert sql (sharding column:" + partColumn + " not provided," + stmt; LOGGER.warn(inf); throw new SQLNonTransientException(inf); } } return null; }
/** * 简单描述该方法的实现功能 * * @param ast QueryTreeNode * @param isSelect 是否是select语句 * @param rrs 数据路由集合 * @param schema 数据库名 the name of datebase * @param ctx ShardingParseInfo(分片) * @param sql 执行sql * @param cachePool * @return 一个数据路由集合 * @throws SQLNonTransientException * @author mycat */ private RouteResultset tryRouteForTables( QueryTreeNode ast, boolean isSelect, RouteResultset rrs, SchemaConfig schema, ShardingParseInfo ctx, String sql, LayerCachePool cachePool) throws SQLNonTransientException { Map<String, TableConfig> tables = schema.getTables(); Map<String, Map<String, Set<ColumnRoutePair>>> tbCondMap = ctx.tablesAndConditions; if (tbCondMap.size() == 1) { // only one table in this sql Map.Entry<String, Map<String, Set<ColumnRoutePair>>> entry = tbCondMap.entrySet().iterator().next(); TableConfig tc = getTableConfig(schema, entry.getKey()); if (tc.getRule() == null && tc.getDataNodes().size() == 1) { rrs.setCacheAble(isSelect); // 20140625 修复 配置为全局表单节点的语句不会自动加上limit sql = addSQLLmit(schema, rrs, ast, sql); return RouterUtil.routeToSingleNode(rrs, tc.getDataNodes().get(0), sql); } Map<String, Set<ColumnRoutePair>> colConds = entry.getValue(); return tryRouteForTable( ast, schema, rrs, isSelect, sql, tc, colConds.get(tc.getPartitionColumn()), colConds, cachePool); } else if (!ctx.joinList.isEmpty()) { for (JoinRel joinRel : ctx.joinList) { TableConfig rootc = schema.getJoinRel2TableMap().get(joinRel.joinSQLExp); if (rootc == null) { if (LOGGER.isDebugEnabled()) { LOGGER.debug( "can't find join relation in schema " + schema.getName() + " :" + joinRel.joinSQLExp + " maybe global table join"); } } else { if (rootc.getName().equals(joinRel.tableA)) { // table a is root table tbCondMap.remove(joinRel.tableB); } else if (rootc.getName().equals(joinRel.tableB)) { // table B is root table tbCondMap.remove(joinRel.tableA); } else if (tbCondMap.containsKey(rootc.getName())) { // contains root table in sql ,then remove all child tbCondMap.remove(joinRel.tableA); tbCondMap.remove(joinRel.tableB); } else { // both there A and B are not root table,remove any // one tbCondMap.remove(joinRel.tableA); } } } } if (tbCondMap.size() > 1) { Set<String> curRNodeSet = new LinkedHashSet<String>(); Set<String> routePairSet = new LinkedHashSet<String>(); // 拆分字段后路由节点 String curTableName = null; Map<String, List<String>> globalTableDataNodesMap = new LinkedHashMap<String, List<String>>(); for (Entry<String, Map<String, Set<ColumnRoutePair>>> e : tbCondMap.entrySet()) { String tableName = e.getKey(); Map<String, Set<ColumnRoutePair>> col2ValMap = e.getValue(); TableConfig tc = tables.get(tableName); if (tc == null) { String msg = "can't find table define in schema " + tableName + " schema:" + schema.getName(); LOGGER.warn(msg); throw new SQLNonTransientException(msg); } else if (tc.getTableType() == TableConfig.TYPE_GLOBAL_TABLE) { // add to globalTablelist globalTableDataNodesMap.put(tc.getName(), tc.getDataNodes()); continue; } Collection<String> newDataNodes = null; String partColmn = tc.getPartitionColumn(); Set<ColumnRoutePair> col2Val = partColmn == null ? null : col2ValMap.get(partColmn); if (col2Val == null || col2Val.isEmpty()) { if (tc.isRuleRequired()) { throw new IllegalArgumentException( "route rule for table " + tableName + " is required: " + sql); } newDataNodes = tc.getDataNodes(); } else { // match table with where condtion of partion colum values newDataNodes = RouterUtil.ruleCalculate(tc, col2Val); } if (curRNodeSet.isEmpty()) { curTableName = tc.getName(); curRNodeSet.addAll(newDataNodes); if (col2Val != null && !col2Val.isEmpty()) { routePairSet.addAll(newDataNodes); } } else { if (col2Val == null || col2Val.isEmpty()) { if (curRNodeSet.retainAll(newDataNodes) && routePairSet.isEmpty()) { String errMsg = "invalid route in sql, multi tables found but datanode has no intersection " + " sql:" + sql; LOGGER.warn(errMsg); throw new SQLNonTransientException(errMsg); } } else { if (routePairSet.isEmpty()) { routePairSet.addAll(newDataNodes); } else if (!checkIfValidMultiTableRoute(routePairSet, newDataNodes) || (curRNodeSet.retainAll(newDataNodes) && routePairSet.isEmpty())) { String errMsg = "invalid route in sql, " + routePairSet + " route to :" + Arrays.toString(routePairSet.toArray()) + " ,but " + tc.getName() + " to " + Arrays.toString(newDataNodes.toArray()) + " sql:" + sql; LOGGER.warn(errMsg); throw new SQLNonTransientException(errMsg); } } // if (!checkIfValidMultiTableRoute(curRNodeSet, // newDataNodes)) { // String errMsg = "invalid route in sql, " + curTableName // + " route to :" // + Arrays.toString(curRNodeSet.toArray()) // + " ,but " + tc.getName() + " to " // + Arrays.toString(newDataNodes.toArray()) // + " sql:" + sql; // LOGGER.warn(errMsg); // throw new SQLNonTransientException(errMsg); // } } } // only global table contains in sql if (!globalTableDataNodesMap.isEmpty() && curRNodeSet.isEmpty()) { List<String> resultList = Lists.newArrayList(); for (List<String> nodeList : globalTableDataNodesMap.values()) { if (resultList.isEmpty()) { resultList = nodeList; } else { if (resultList.retainAll(nodeList) && resultList.isEmpty()) { String errMsg = "invalid route in sql, multi global tables found but datanode has no intersection " + " sql:" + sql; LOGGER.warn(errMsg); throw new SQLNonTransientException(errMsg); } } } if (resultList.size() == 1) { rrs.setCacheAble(true); sql = addSQLLmit(schema, rrs, ast, sql); rrs = RouterUtil.routeToSingleNode(rrs, resultList.get(0), sql); } else { // mulit routes ,not cache route result rrs.setCacheAble(false); rrs = RouterUtil.routeToSingleNode(rrs, getRandomDataNode(resultList), sql); } return rrs; } else if (!globalTableDataNodesMap.isEmpty() && !curRNodeSet.isEmpty()) { // judge if global table contains all dataNodes of other tables for (Map.Entry<String, List<String>> entry : globalTableDataNodesMap.entrySet()) { if (!entry.getValue().containsAll(curRNodeSet)) { String errMsg = "invalid route in sql, " + curTableName + " route to :" + Arrays.toString(curRNodeSet.toArray()) + " ,but " + entry.getKey() + " to " + Arrays.toString(entry.getValue().toArray()) + " sql:" + sql; LOGGER.warn(errMsg); throw new SQLNonTransientException(errMsg); } } } if (curRNodeSet.size() > 1) { LOGGER.warn( "multi route tables found in this sql ,tables:" + Arrays.toString(tbCondMap.keySet().toArray()) + " sql:" + sql); return routeToMultiNode(schema, isSelect, isSelect, ast, rrs, curRNodeSet, sql); } else { return RouterUtil.routeToSingleNode(rrs, curRNodeSet.iterator().next(), sql); } } else { // only one table Map.Entry<String, Map<String, Set<ColumnRoutePair>>> entry = tbCondMap.entrySet().iterator().next(); Map<String, Set<ColumnRoutePair>> allColValues = entry.getValue(); TableConfig tc = getTableConfig(schema, entry.getKey()); return tryRouteForTable( ast, schema, rrs, isSelect, sql, tc, allColValues.get(tc.getPartitionColumn()), allColValues, cachePool); } }
/** * 简单描述该方法的实现功能 * * @param ast QueryTreeNode * @param schema 数据库名 * @param rrs 数据路由集合 * @param isSelect 是否是select语句标志 * @param sql 执行语句 * @param tc 表实体 * @param ruleCol2Val 一个ColumnRoutePair集合 * @param allColConds 一个ColumnRoutePair集合 * @param cachePool * @return 一个数据路由集合 * @throws SQLNonTransientException * @author mycat */ private RouteResultset tryRouteForTable( QueryTreeNode ast, SchemaConfig schema, RouteResultset rrs, boolean isSelect, String sql, TableConfig tc, Set<ColumnRoutePair> ruleCol2Val, Map<String, Set<ColumnRoutePair>> allColConds, LayerCachePool cachePool) throws SQLNonTransientException { if (tc.getTableType() == TableConfig.TYPE_GLOBAL_TABLE && isSelect) { sql = addSQLLmit(schema, rrs, ast, sql); return RouterUtil.routeToSingleNode(rrs, tc.getRandomDataNode(), sql); } // no partion define or no where condtion for this table or no // partion column condtions boolean cache = isSelect; if (ruleCol2Val == null || ruleCol2Val.isEmpty()) { if (tc.isRuleRequired()) { throw new IllegalArgumentException( "route rule for table " + tc.getName() + " is required: " + sql); } else if (allColConds != null && allColConds.size() == 1) { // try if can route by ER relation if (tc.isSecondLevel() && tc.getParentTC().getPartitionColumn().equals(tc.getParentKey())) { Set<ColumnRoutePair> joinKeyPairs = allColConds.get(tc.getJoinKey()); if (joinKeyPairs != null) { Set<String> dataNodeSet = RouterUtil.ruleCalculate(tc.getParentTC(), joinKeyPairs); if (dataNodeSet.isEmpty()) { throw new SQLNonTransientException("parent key can't find any valid datanode "); } if (LOGGER.isDebugEnabled()) { LOGGER.debug( "found partion nodes (using parent partion rule directly) for child table to update " + Arrays.toString(dataNodeSet.toArray()) + " sql :" + sql); } if (dataNodeSet.size() > 1) { return routeToMultiNode(schema, isSelect, isSelect, ast, rrs, dataNodeSet, sql); } else { rrs.setCacheAble(true); return RouterUtil.routeToSingleNode(rrs, dataNodeSet.iterator().next(), sql); } } } // try by primary key if found in cache Set<ColumnRoutePair> primaryKeyPairs = allColConds.get(tc.getPrimaryKey()); if (primaryKeyPairs != null) { if (LOGGER.isDebugEnabled()) { LOGGER.debug("try to find cache by primary key "); } cache = false; Set<String> dataNodes = new HashSet<String>(primaryKeyPairs.size()); boolean allFound = true; String tableKey = schema.getName() + '_' + tc.getName(); for (ColumnRoutePair pair : primaryKeyPairs) { String cacheKey = pair.colValue; String dataNode = (String) cachePool.get(tableKey, cacheKey); if (dataNode == null) { allFound = false; break; } else { dataNodes.add(dataNode); } } if (allFound) { return routeToMultiNode(schema, isSelect, isSelect, ast, rrs, dataNodes, sql); } // need cache primary key ->datanode relation if (isSelect && tc.getPrimaryKey() != null) { rrs.setPrimaryKey(tableKey + '.' + tc.getPrimaryKey()); } } } return routeToMultiNode(schema, isSelect, cache, ast, rrs, tc.getDataNodes(), sql); } // match table with where condtion of partion colum values Set<String> dataNodeSet = RouterUtil.ruleCalculate(tc, ruleCol2Val); if (dataNodeSet.size() == 1) { rrs.setCacheAble(isSelect); return RouterUtil.routeToSingleNode(rrs, dataNodeSet.iterator().next(), sql); } else { return routeToMultiNode(schema, isSelect, isSelect, ast, rrs, dataNodeSet, sql); } }