public static void shutdown() {
logger_.info("Shutting down ...");
synchronized (MessagingService.class) {
/* Stop listening on any socket */
for (SelectionKey skey : listenSockets_.values()) {
SelectorManager.getSelectorManager().cancel(skey);
}
listenSockets_.clear();
/* Shutdown the threads in the EventQueue's */
messageDeserializationExecutor_.shutdownNow();
messageSerializerExecutor_.shutdownNow();
messageDeserializerExecutor_.shutdownNow();
streamExecutor_.shutdownNow();
/* shut down the cachetables */
taskCompletionMap_.shutdown();
callbackMap_.shutdown();
/* Interrupt the selector manager thread */
SelectorManager.getSelectorManager().interrupt();
poolTable_.clear();
verbHandlers_.clear();
bShutdown_ = true;
}
logger_.debug("Shutdown invocation complete.");
}
public NonBlockingIdentityHashMap<Long, TestKey> getMapMultithreaded()
throws InterruptedException, ExecutionException {
final int threadCount = _items.keySet().size();
final NonBlockingIdentityHashMap<Long, TestKey> map =
new NonBlockingIdentityHashMap<Long, TestKey>();
// use a barrier to open the gate for all threads at once to avoid rolling start and no actual
// concurrency
final CyclicBarrier barrier = new CyclicBarrier(threadCount);
final ExecutorService ex = Executors.newFixedThreadPool(threadCount);
final CompletionService<Integer> co = new ExecutorCompletionService<Integer>(ex);
for (Integer type : _items.keySet()) {
// A linked-list of things to insert
List<TestKey> items = _items.get(type);
TestKeyFeederThread feeder = new TestKeyFeederThread(type, items, map, barrier);
co.submit(feeder);
}
// wait for all threads to return
int itemCount = 0;
for (int retCount = 0; retCount < threadCount; retCount++) {
final Future<Integer> result = co.take();
itemCount += result.get();
}
ex.shutdown();
return map;
}
// Concurrent insertion & then iterator test.
public static void testNonBlockingIdentityHashMapIterator() throws InterruptedException {
final int ITEM_COUNT1 = 1000;
final int THREAD_COUNT = 5;
final int PER_CNT = ITEM_COUNT1 / THREAD_COUNT;
final int ITEM_COUNT = PER_CNT * THREAD_COUNT; // fix roundoff for odd thread counts
NonBlockingIdentityHashMap<Long, TestKey> nbhml =
new NonBlockingIdentityHashMap<Long, TestKey>();
// use a barrier to open the gate for all threads at once to avoid rolling
// start and no actual concurrency
final CyclicBarrier barrier = new CyclicBarrier(THREAD_COUNT);
final ExecutorService ex = Executors.newFixedThreadPool(THREAD_COUNT);
final CompletionService<Object> co = new ExecutorCompletionService<Object>(ex);
for (int i = 0; i < THREAD_COUNT; i++) {
co.submit(new NBHMLFeeder(nbhml, PER_CNT, barrier, i * PER_CNT));
}
for (int retCount = 0; retCount < THREAD_COUNT; retCount++) {
co.take();
}
ex.shutdown();
assertEquals("values().size()", ITEM_COUNT, nbhml.values().size());
assertEquals("entrySet().size()", ITEM_COUNT, nbhml.entrySet().size());
int itemCount = 0;
for (TestKey K : nbhml.values()) itemCount++;
assertEquals("values().iterator() count", ITEM_COUNT, itemCount);
}
/**
* Test concurrent reader and writer (GH-458).
*
* <p><b>Test case:</b>
*
* <ol>
* <li/>start a long running reader;
* <li/>try to start a writer: it should time out;
* <li/>stop the reader;
* <li/>start the writer again: it should succeed.
* </ol>
*
* @throws Exception error during request execution
*/
@Test
@Ignore("There is no way to stop a query on the server!")
public void testReaderWriter() throws Exception {
final String readerQuery = "?query=(1%20to%20100000000000000)%5b.=1%5d";
final String writerQuery = "/test.xml";
final byte[] content = Token.token("<a/>");
final Get readerAction = new Get(readerQuery);
final Put writerAction = new Put(writerQuery, content);
final ExecutorService exec = Executors.newFixedThreadPool(2);
// start reader
exec.submit(readerAction);
Performance.sleep(TIMEOUT); // delay in order to be sure that the reader has started
// start writer
Future<HTTPResponse> writer = exec.submit(writerAction);
try {
final HTTPResponse result = writer.get(TIMEOUT, TimeUnit.MILLISECONDS);
if (result.status.isSuccess()) fail("Database modified while a reader is running");
throw new Exception(result.toString());
} catch (final TimeoutException e) {
// writer is blocked by the reader: stop it
writerAction.stop = true;
}
// stop reader
readerAction.stop = true;
// start the writer again
writer = exec.submit(writerAction);
assertEquals(HTTPCode.CREATED, writer.get().status);
}
public static void main(String[] args) {
ExecutorService pool = Executors.newFixedThreadPool(2);
final Vector[] vecs = {
new Vector(), new Vector(),
};
vecs[0].add(vecs[1]);
vecs[1].add(vecs[0]);
for (int i = 0; i < 2; i++) {
final int threadNumber = i;
pool.submit(
new Callable() {
public Object call() throws Exception {
for (int i = 0; i < 1000 * 1000; i++) {
ObjectOutputStream out = new ObjectOutputStream(new NullOutputStream());
out.writeObject(vecs[threadNumber]);
out.close();
}
System.out.println("done");
return null;
}
});
}
}
@Override
public Boolean call() throws Exception {
long timeoutMillis = 5000;
try {
getServersFile();
getZkRunning();
while (true) {
while (!restartQueue.isEmpty()) {
LOG.debug("Restart queue size [" + restartQueue.size() + "]");
RestartHandler handler = restartQueue.poll();
Future<ScriptContext> runner = pool.submit(handler);
ScriptContext scriptContext = runner.get(); // blocking call
if (scriptContext.getExitCode() != 0) restartQueue.add(handler);
}
try {
Thread.sleep(timeoutMillis);
} catch (InterruptedException e) {
}
}
} catch (Exception e) {
e.printStackTrace();
LOG.error(e);
pool.shutdown();
throw e;
}
}
private static void build(Iterable<String> in) throws IOException, InterruptedException {
final List<String> args = newArrayList(in);
Collections.sort(args);
final int threads = Runtime.getRuntime().availableProcessors();
final ExecutorService ex = Executors.newFixedThreadPool(threads);
final String base = "base";
new WithVm("base").createIfNotPresent();
for (String pkg : args)
ex.submit(
() -> {
final WithVm newVm = new WithVm("fbuild-" + pkg, TimeUnit.MINUTES.toMillis(30));
final File rbuild = new File("wip-" + pkg + ".rbuild");
try {
newVm.cloneFrom(base);
newVm.start();
newVm.inTee(rbuild, "apt-get", "-oAPT::Get::Only-Source=true", "source", pkg);
newVm.inTee(rbuild, "apt-get", "build-dep", "-y", "--force-yes", pkg);
newVm.inTee(rbuild, "ifdown", "eth0");
final boolean success =
0
== newVm.inTee(
rbuild, "sh", "-c", "cd " + pkg + "-* && dpkg-buildpackage -us -uc");
newVm.stopNow();
if (success) {
rbuild.renameTo(new File("success-" + pkg + ".rbuild"));
newVm.destroy();
System.out.println("success: " + pkg);
} else {
rbuild.renameTo(new File("failure-" + pkg + ".rbuild"));
System.out.println("failure: " + pkg);
}
} catch (Exception e) {
rbuild.renameTo(new File("error-" + pkg + ".rbuild"));
System.err.println("build error: " + pkg);
e.printStackTrace();
newVm.stopNow();
}
return null;
});
ex.shutdown();
ex.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
}
/** {@inheritDoc} */
@Override
protected void maybeStartStream() throws IOException {
// connector
final StreamConnector connector = getStreamConnector();
if (connector == null) return;
synchronized (this) {
if (started) return;
threadPool.execute(
new Runnable() {
@Override
public void run() {
try {
Sctp.init();
runOnDtlsTransport(connector);
} catch (IOException e) {
logger.error(e, e);
} finally {
try {
Sctp.finish();
} catch (IOException e) {
logger.error("Failed to shutdown SCTP stack", e);
}
}
}
});
started = true;
}
}
public static void main(final String[] args) throws Exception {
final int length = (1 << 14);
final ICircularLongsBuffer buffer = new UnsafeCircularLongsBuffer(length);
final File file = new File("output.log");
log.info("File " + file);
file.delete();
// file.deleteOnExit();
final FastLoggerImpl logger =
new FastLoggerImpl(
new ThreadFactory() {
@Override
public Thread newThread(final Runnable r) {
return new Thread(r);
}
},
buffer,
WaitingStrategy.SPINNING,
new RawFileWriter(file));
logger.startDraining();
final int workers = Runtime.getRuntime().availableProcessors() - 1;
final ExecutorService workersPool = Executors.newFixedThreadPool(workers);
for (int i = 0; i < workers; i++) {
workersPool.submit(
new Runnable() {
@Override
public void run() {
try {
for (int i = 0; i < 100000000; i++) {
logger.log("Message %f -- %d ").with(25.98 + i).with(100 - i).submit();
}
} catch (Exception e) {
log.error("Error", e);
}
}
});
}
workersPool.shutdown();
workersPool.awaitTermination(1000, TimeUnit.SECONDS);
log.info("Finished");
System.exit(1);
}
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
if (attrs.isRegularFile()) {
if (m != null) m.reset(root.relativize(file).toString());
if (m == null || m.matches())
futures.add(exec.submit(new FileLoader(root, file, blocSize)));
}
return FileVisitResult.CONTINUE;
}
static Future<String> futureOutputOf(final InputStream is) {
return drainers.submit(
new Callable<String>() {
public String call() throws IOException {
return outputOf(is);
}
});
}
public static void start() {
if (started) return;
// start code
pool = Executors.newFixedThreadPool(poolSize);
System.out.println("downloader start!!");
for (DownloadTask task : tasks) pool.execute(task);
started = true;
}
/*
Use this version for fire and forget style messaging.
*/
public void sendOneWay(Message message, EndPoint to) {
// do local deliveries
if (message.getFrom().equals(to)) {
MessagingService.receive(message);
return;
}
Runnable tcpWriteEvent = new MessageSerializationTask(message, to);
messageSerializerExecutor_.execute(tcpWriteEvent);
}
public static void main(String[] args) throws Exception {
HttpServer s1 = null;
HttpsServer s2 = null;
ExecutorService executor = null;
try {
String root = System.getProperty("test.src") + "/docs";
System.out.print("Test12: ");
InetSocketAddress addr = new InetSocketAddress(0);
s1 = HttpServer.create(addr, 0);
s2 = HttpsServer.create(addr, 0);
HttpHandler h = new FileServerHandler(root);
HttpContext c1 = s1.createContext("/test1", h);
HttpContext c2 = s2.createContext("/test1", h);
executor = Executors.newCachedThreadPool();
s1.setExecutor(executor);
s2.setExecutor(executor);
ctx = new SimpleSSLContext(System.getProperty("test.src")).get();
s2.setHttpsConfigurator(new HttpsConfigurator(ctx));
s1.start();
s2.start();
int port = s1.getAddress().getPort();
int httpsport = s2.getAddress().getPort();
Runner r[] = new Runner[8];
r[0] = new Runner(true, "http", root + "/test1", port, "smallfile.txt", 23);
r[1] = new Runner(true, "http", root + "/test1", port, "largefile.txt", 2730088);
r[2] = new Runner(true, "https", root + "/test1", httpsport, "smallfile.txt", 23);
r[3] = new Runner(true, "https", root + "/test1", httpsport, "largefile.txt", 2730088);
r[4] = new Runner(false, "http", root + "/test1", port, "smallfile.txt", 23);
r[5] = new Runner(false, "http", root + "/test1", port, "largefile.txt", 2730088);
r[6] = new Runner(false, "https", root + "/test1", httpsport, "smallfile.txt", 23);
r[7] = new Runner(false, "https", root + "/test1", httpsport, "largefile.txt", 2730088);
start(r);
join(r);
System.out.println("OK");
} finally {
delay();
if (s1 != null) s1.stop(2);
if (s2 != null) s2.stop(2);
if (executor != null) executor.shutdown();
}
}
private static void enqueueRunnable(String stageName, Runnable runnable) {
IStage stage = StageManager.getStage(stageName);
if (stage != null) {
logger_.info("Running on stage " + stage.getName());
stage.execute(runnable);
} else {
logger_.info("Running on default stage - beware");
messageSerializerExecutor_.execute(runnable);
}
}
/**
* Test 2 concurrent readers (GH-458).
*
* <p><b>Test case:</b>
*
* <ol>
* <li/>start a long running reader;
* <li/>start a fast reader: it should succeed.
* </ol>
*
* @throws Exception error during request execution
*/
@Test
public void testMultipleReaders() throws Exception {
final String number = "63177";
final String slowQuery = "?query=(1%20to%20100000000000000)%5b.=1%5d";
final String fastQuery = "?query=" + number;
final Get slowAction = new Get(slowQuery);
final Get fastAction = new Get(fastQuery);
final ExecutorService exec = Executors.newFixedThreadPool(2);
exec.submit(slowAction);
Performance.sleep(TIMEOUT); // delay in order to be sure that the reader has started
final Future<HTTPResponse> fast = exec.submit(fastAction);
try {
final HTTPResponse result = fast.get(TIMEOUT, TimeUnit.MILLISECONDS);
assertEquals(HTTPCode.OK, result.status);
assertEquals(number, result.data);
} finally {
slowAction.stop = true;
}
}
public Client() {
try {
Socket socket = new Socket("192.168.1.34", PORT);
exec.execute(new SendMsg(socket));
System.out.println("[" + socket.getInetAddress() + "] ' Joining ... ");
BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String msg;
while ((msg = br.readLine()) != null) {
System.out.println(msg);
}
} catch (IOException e) {
}
}
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
// get the number of workers to run
int count = this.getRequestedCount(request, 4);
// get the executor service
final ServletContext sc = this.getServletContext();
final ExecutorService executorService = (ExecutorService) sc.getAttribute("myExecutorService");
// create work coordinator
CountDownLatch countDownLatch = new CountDownLatch(count);
Long t1 = System.nanoTime();
// create the workers
List<RunnableWorkUnit2> workers = new ArrayList<RunnableWorkUnit2>();
for (int i = 0; i < count; i++) {
RunnableWorkUnit2 wu =
new RunnableWorkUnit2("RunnableTask" + String.valueOf(i + 1), countDownLatch);
workers.add(wu);
}
// run the workers through the executor
for (RunnableWorkUnit2 wu : workers) {
executorService.execute(wu);
}
try {
System.out.println("START WAITING");
countDownLatch.await();
System.out.println("DONE WAITING");
} catch (InterruptedException ex) {
ex.printStackTrace();
}
Long t2 = System.nanoTime();
Writer w = response.getWriter();
w.write(String.format("\n Request processed in %dms!", (t2 - t1) / 1000000));
w.flush();
w.close();
}
/**
* Test concurrent writers (GH-458).
*
* <p><b>Test case:</b>
*
* <ol>
* <li/>start several writers one after another;
* <li/>all writers should succeed.
* </ol>
*
* @throws Exception error during request execution
*/
@Test
public void testMultipleWriters() throws Exception {
final int count = 10;
final String template =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<command xmlns=\"http://basex.org/rest\"><text><![CDATA["
+ "ADD TO %1$d <node id=\"%1$d\"/>"
+ "]]></text></command>";
@SuppressWarnings("unchecked")
final Future<HTTPResponse>[] tasks = new Future[count];
final ExecutorService exec = Executors.newFixedThreadPool(count);
// start all writers (not at the same time, but still in parallel)
for (int i = 0; i < count; i++) {
final String command = String.format(template, i);
tasks[i] = exec.submit(new Post("", Token.token(command)));
}
// check if all have finished successfully
for (final Future<HTTPResponse> task : tasks) {
assertEquals(HTTPCode.OK, task.get(TIMEOUT, TimeUnit.MILLISECONDS).status);
}
}
public static void main(String[] args) throws Exception {
try {
lock = new Object();
if (args.length > 0) {
NUM = Integer.parseInt(args[0]);
}
execs = Executors.newFixedThreadPool(5);
httpServer = createHttpServer(execs);
port = httpServer.getAddress().getPort();
pool = Executors.newFixedThreadPool(10);
httpServer.start();
for (int i = 0; i < NUM; i++) {
pool.execute(new Client());
if (error) {
throw new Exception("error in test");
}
}
System.out.println("Main thread waiting");
pool.shutdown();
long latest = System.currentTimeMillis() + 200 * 1000;
while (System.currentTimeMillis() < latest) {
if (pool.awaitTermination(2000L, TimeUnit.MILLISECONDS)) {
System.out.println("Main thread done!");
return;
}
if (error) {
throw new Exception("error in test");
}
}
throw new Exception("error in test: timed out");
} finally {
httpServer.stop(0);
pool.shutdownNow();
execs.shutdownNow();
}
}
public void run() {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
PrintWriter pw = new PrintWriter(socket.getOutputStream(), true);
String msg;
while (true) {
msg = br.readLine();
pw.println(msg);
if (msg.trim().equals("-quit")) {
pw.close();
br.close();
exec.shutdownNow();
break;
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
static void realMain(String[] args) throws Throwable {
// jmap doesn't work on Windows
if (System.getProperty("os.name").startsWith("Windows")) return;
final String childClassName = Job.class.getName();
final String classToCheckForLeaks = Job.classToCheckForLeaks();
final String uniqueID = String.valueOf(new Random().nextInt(Integer.MAX_VALUE));
final String[] jobCmd = {
java,
"-Xmx8m",
"-classpath",
System.getProperty("test.classes", "."),
childClassName,
uniqueID
};
final Process p = new ProcessBuilder(jobCmd).start();
final String childPid =
match(
commandOutputOf(jps, "-m"),
"(?m)^ *([0-9]+) +\\Q" + childClassName + "\\E *" + uniqueID + "$",
1);
final int n0 = objectsInUse(p, childPid, classToCheckForLeaks);
final int n1 = objectsInUse(p, childPid, classToCheckForLeaks);
equal(p.waitFor(), 0);
equal(p.exitValue(), 0);
failed += p.exitValue();
// Check that no objects were leaked.
System.out.printf("%d -> %d%n", n0, n1);
check(Math.abs(n1 - n0) < 2); // Almost always n0 == n1
check(n1 < 20);
drainers.shutdown();
}
/**
* Sets the simulcast layers for this receiver and fires an event about it.
*
* @param simulcastLayers the simulcast layers for this receiver.
*/
public void setSimulcastLayers(SimulcastLayer[] simulcastLayers) {
this.simulcastLayers = simulcastLayers;
if (logger.isInfoEnabled()) {
if (simulcastLayers == null) {
logInfo("Simulcast disabled.");
} else {
for (SimulcastLayer l : simulcastLayers) {
logInfo(l.getOrder() + ": " + l.getPrimarySSRC());
}
}
}
executorService.execute(
new Runnable() {
public void run() {
firePropertyChange(SIMULCAST_LAYERS_PNAME, null, null);
}
});
// TODO If simulcastLayers has changed, then simulcastLayerFrameHistory
// has very likely become irrelevant. In other words, clear
// simulcastLayerFrameHistory.
}
private void runOnDtlsTransport(StreamConnector connector) throws IOException {
DtlsControlImpl dtlsControl = (DtlsControlImpl) getTransportManager().getDtlsControl(this);
DtlsTransformEngine engine = dtlsControl.getTransformEngine();
final DtlsPacketTransformer transformer = (DtlsPacketTransformer) engine.getRTPTransformer();
byte[] receiveBuffer = new byte[SCTP_BUFFER_SIZE];
if (LOG_SCTP_PACKETS) {
System.setProperty(
ConfigurationService.PNAME_SC_HOME_DIR_LOCATION, System.getProperty("java.io.tmpdir"));
System.setProperty(
ConfigurationService.PNAME_SC_HOME_DIR_NAME, SctpConnection.class.getName());
}
synchronized (this) {
// FIXME local SCTP port is hardcoded in bridge offer SDP (Jitsi
// Meet)
sctpSocket = Sctp.createSocket(5000);
assocIsUp = false;
acceptedIncomingConnection = false;
}
// Implement output network link for SCTP stack on DTLS transport
sctpSocket.setLink(
new NetworkLink() {
@Override
public void onConnOut(SctpSocket s, byte[] packet) throws IOException {
if (LOG_SCTP_PACKETS) {
LibJitsi.getPacketLoggingService()
.logPacket(
PacketLoggingService.ProtocolName.ICE4J,
new byte[] {0, 0, 0, (byte) debugId},
5000,
new byte[] {0, 0, 0, (byte) (debugId + 1)},
remoteSctpPort,
PacketLoggingService.TransportName.UDP,
true,
packet);
}
// Send through DTLS transport
transformer.sendApplicationData(packet, 0, packet.length);
}
});
if (logger.isDebugEnabled()) {
logger.debug("Connecting SCTP to port: " + remoteSctpPort + " to " + getEndpoint().getID());
}
sctpSocket.setNotificationListener(this);
sctpSocket.listen();
// FIXME manage threads
threadPool.execute(
new Runnable() {
@Override
public void run() {
SctpSocket sctpSocket = null;
try {
// sctpSocket is set to null on close
sctpSocket = SctpConnection.this.sctpSocket;
while (sctpSocket != null) {
if (sctpSocket.accept()) {
acceptedIncomingConnection = true;
break;
}
Thread.sleep(100);
sctpSocket = SctpConnection.this.sctpSocket;
}
if (isReady()) {
notifySctpConnectionReady();
}
} catch (Exception e) {
logger.error("Error accepting SCTP connection", e);
}
if (sctpSocket == null && logger.isInfoEnabled()) {
logger.info(
"SctpConnection " + getID() + " closed" + " before SctpSocket accept()-ed.");
}
}
});
// Notify that from now on SCTP connection is considered functional
sctpSocket.setDataCallback(this);
// Setup iceSocket
DatagramSocket datagramSocket = connector.getDataSocket();
if (datagramSocket != null) {
this.iceSocket = new IceUdpSocketWrapper(datagramSocket);
} else {
this.iceSocket = new IceTcpSocketWrapper(connector.getDataTCPSocket());
}
DatagramPacket rcvPacket = new DatagramPacket(receiveBuffer, 0, receiveBuffer.length);
// Receive loop, breaks when SCTP socket is closed
try {
do {
iceSocket.receive(rcvPacket);
RawPacket raw =
new RawPacket(rcvPacket.getData(), rcvPacket.getOffset(), rcvPacket.getLength());
raw = transformer.reverseTransform(raw);
// Check for app data
if (raw == null) continue;
if (LOG_SCTP_PACKETS) {
LibJitsi.getPacketLoggingService()
.logPacket(
PacketLoggingService.ProtocolName.ICE4J,
new byte[] {0, 0, 0, (byte) (debugId + 1)},
remoteSctpPort,
new byte[] {0, 0, 0, (byte) debugId},
5000,
PacketLoggingService.TransportName.UDP,
false,
raw.getBuffer(),
raw.getOffset(),
raw.getLength());
}
// Pass network packet to SCTP stack
sctpSocket.onConnIn(raw.getBuffer(), raw.getOffset(), raw.getLength());
} while (true);
} finally {
// Eventually, close the socket although it should happen from
// expire().
synchronized (this) {
assocIsUp = false;
acceptedIncomingConnection = false;
if (sctpSocket != null) {
sctpSocket.close();
sctpSocket = null;
}
}
}
}
/**
* 添加一个下载任务，如果Downloader.start()已经启动，那么将该任务提交到线程池中去
*
* @param task
*/
static void add(DownloadTask task) {
tasks.add(task);
if (started) pool.execute(task);
}
public void close() {
if (executor != null && executor instanceof ExecutorService)
((ExecutorService) executor).shutdown();
}
protected void handleCommand(String command) {
if (command.contains("__")) {
namespace = command.split("__")[0];
command = command.substring(command.indexOf("__") + 2);
}
if (echo) {
if (Thread.currentThread().getName().toLowerCase().indexOf("main") < 0)
println(" [" + Thread.currentThread().getName() + "] " + command);
else println(command);
}
if (command == null || command.startsWith("//")) return;
command = command.trim();
if (command == null || command.length() == 0) {
return;
}
String first = command;
int spaceIndex = command.indexOf(' ');
String[] argsSplit = command.split(" ");
String[] args = new String[argsSplit.length];
for (int i = 0; i < argsSplit.length; i++) {
args[i] = argsSplit[i].trim();
}
if (spaceIndex != -1) {
first = args[0];
}
if (command.startsWith("help")) {
handleHelp(command);
} else if (first.startsWith("#") && first.length() > 1) {
int repeat = Integer.parseInt(first.substring(1));
long t0 = Clock.currentTimeMillis();
for (int i = 0; i < repeat; i++) {
handleCommand(command.substring(first.length()).replaceAll("\\$i", "" + i));
}
println("ops/s = " + repeat * 1000 / (Clock.currentTimeMillis() - t0));
return;
} else if (first.startsWith("&") && first.length() > 1) {
final int fork = Integer.parseInt(first.substring(1));
ExecutorService pool = Executors.newFixedThreadPool(fork);
final String threadCommand = command.substring(first.length());
for (int i = 0; i < fork; i++) {
final int threadID = i;
pool.submit(
new Runnable() {
public void run() {
String command = threadCommand;
String[] threadArgs = command.replaceAll("\\$t", "" + threadID).trim().split(" ");
// TODO &t #4 m.putmany x k
if ("m.putmany".equals(threadArgs[0]) || "m.removemany".equals(threadArgs[0])) {
if (threadArgs.length < 4) {
command += " " + Integer.parseInt(threadArgs[1]) * threadID;
}
}
handleCommand(command);
}
});
}
pool.shutdown();
try {
pool.awaitTermination(60 * 60, TimeUnit.SECONDS); // wait 1h
} catch (Exception e) {
e.printStackTrace();
}
} else if (first.startsWith("@")) {
if (first.length() == 1) {
println("usage: @<file-name>");
return;
}
File f = new File(first.substring(1));
println("Executing script file " + f.getAbsolutePath());
if (f.exists()) {
try {
BufferedReader br = new BufferedReader(new FileReader(f));
String l = br.readLine();
while (l != null) {
handleCommand(l);
l = br.readLine();
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
} else {
println("File not found! " + f.getAbsolutePath());
}
} else if (command.indexOf(';') != -1) {
StringTokenizer st = new StringTokenizer(command, ";");
while (st.hasMoreTokens()) {
handleCommand(st.nextToken());
}
return;
} else if ("silent".equals(first)) {
silent = Boolean.parseBoolean(args[1]);
} else if ("shutdown".equals(first)) {
hazelcast.getLifecycleService().shutdown();
} else if ("echo".equals(first)) {
echo = Boolean.parseBoolean(args[1]);
println("echo: " + echo);
} else if ("ns".equals(first)) {
if (args.length > 1) {
namespace = args[1];
println("namespace: " + namespace);
// init();
}
} else if ("whoami".equals(first)) {
println(hazelcast.getCluster().getLocalMember());
} else if ("who".equals(first)) {
println(hazelcast.getCluster());
} else if ("jvm".equals(first)) {
System.gc();
println("Memory max: " + Runtime.getRuntime().maxMemory() / 1024 / 1024 + "M");
println(
"Memory free: "
+ Runtime.getRuntime().freeMemory() / 1024 / 1024
+ "M "
+ (int) (Runtime.getRuntime().freeMemory() * 100 / Runtime.getRuntime().maxMemory())
+ "%");
long total = Runtime.getRuntime().totalMemory();
long free = Runtime.getRuntime().freeMemory();
println("Used Memory:" + ((total - free) / 1024 / 1024) + "MB");
println("# procs: " + Runtime.getRuntime().availableProcessors());
println(
"OS info: "
+ ManagementFactory.getOperatingSystemMXBean().getArch()
+ " "
+ ManagementFactory.getOperatingSystemMXBean().getName()
+ " "
+ ManagementFactory.getOperatingSystemMXBean().getVersion());
println(
"JVM: "
+ ManagementFactory.getRuntimeMXBean().getVmVendor()
+ " "
+ ManagementFactory.getRuntimeMXBean().getVmName()
+ " "
+ ManagementFactory.getRuntimeMXBean().getVmVersion());
} else if (first.indexOf("ock") != -1 && first.indexOf(".") == -1) {
handleLock(args);
} else if (first.indexOf(".size") != -1) {
handleSize(args);
} else if (first.indexOf(".clear") != -1) {
handleClear(args);
} else if (first.indexOf(".destroy") != -1) {
handleDestroy(args);
} else if (first.indexOf(".iterator") != -1) {
handleIterator(args);
} else if (first.indexOf(".contains") != -1) {
handleContains(args);
} else if (first.indexOf(".stats") != -1) {
handStats(args);
} else if ("t.publish".equals(first)) {
handleTopicPublish(args);
} else if ("q.offer".equals(first)) {
handleQOffer(args);
} else if ("q.take".equals(first)) {
handleQTake(args);
} else if ("q.poll".equals(first)) {
handleQPoll(args);
} else if ("q.peek".equals(first)) {
handleQPeek(args);
} else if ("q.capacity".equals(first)) {
handleQCapacity(args);
} else if ("q.offermany".equals(first)) {
handleQOfferMany(args);
} else if ("q.pollmany".equals(first)) {
handleQPollMany(args);
} else if ("s.add".equals(first)) {
handleSetAdd(args);
} else if ("s.remove".equals(first)) {
handleSetRemove(args);
} else if ("s.addmany".equals(first)) {
handleSetAddMany(args);
} else if ("s.removemany".equals(first)) {
handleSetRemoveMany(args);
} else if (first.equals("m.replace")) {
handleMapReplace(args);
} else if (first.equalsIgnoreCase("m.putIfAbsent")) {
handleMapPutIfAbsent(args);
} else if (first.equals("m.putAsync")) {
handleMapPutAsync(args);
} else if (first.equals("m.getAsync")) {
handleMapGetAsync(args);
} else if (first.equals("m.put")) {
handleMapPut(args);
} else if (first.equals("m.get")) {
handleMapGet(args);
} else if (first.equalsIgnoreCase("m.getMapEntry")) {
handleMapGetMapEntry(args);
} else if (first.equals("m.remove")) {
handleMapRemove(args);
} else if (first.equals("m.evict")) {
handleMapEvict(args);
} else if (first.equals("m.putmany") || first.equalsIgnoreCase("m.putAll")) {
handleMapPutMany(args);
} else if (first.equals("m.getmany")) {
handleMapGetMany(args);
} else if (first.equals("m.removemany")) {
handleMapRemoveMany(args);
} else if (command.equalsIgnoreCase("m.localKeys")) {
handleMapLocalKeys();
} else if (command.equals("m.keys")) {
handleMapKeys();
} else if (command.equals("m.values")) {
handleMapValues();
} else if (command.equals("m.entries")) {
handleMapEntries();
} else if (first.equals("m.lock")) {
handleMapLock(args);
} else if (first.equalsIgnoreCase("m.tryLock")) {
handleMapTryLock(args);
} else if (first.equals("m.unlock")) {
handleMapUnlock(args);
} else if (first.indexOf(".addListener") != -1) {
handleAddListener(args);
} else if (first.equals("m.removeMapListener")) {
handleRemoveListener(args);
} else if (first.equals("m.unlock")) {
handleMapUnlock(args);
} else if (first.equals("l.add")) {
handleListAdd(args);
} else if (first.equals("l.set")) {
handleListSet(args);
} else if ("l.addmany".equals(first)) {
handleListAddMany(args);
} else if (first.equals("l.remove")) {
handleListRemove(args);
} else if (first.equals("l.contains")) {
handleListContains(args);
} else if ("a.get".equals(first)) {
handleAtomicNumberGet(args);
} else if ("a.set".equals(first)) {
handleAtomicNumberSet(args);
} else if ("a.inc".equals(first)) {
handleAtomicNumberInc(args);
} else if ("a.dec".equals(first)) {
handleAtomicNumberDec(args);
// } else if (first.equals("execute")) {
// execute(args);
} else if (first.equals("partitions")) {
handlePartitions(args);
// } else if (first.equals("txn")) {
// hazelcast.getTransaction().begin();
// } else if (first.equals("commit")) {
// hazelcast.getTransaction().commit();
// } else if (first.equals("rollback")) {
// hazelcast.getTransaction().rollback();
// } else if (first.equalsIgnoreCase("executeOnKey")) {
// executeOnKey(args);
// } else if (first.equalsIgnoreCase("executeOnMember")) {
// executeOnMember(args);
// } else if (first.equalsIgnoreCase("executeOnMembers")) {
// executeOnMembers(args);
// } else if (first.equalsIgnoreCase("longOther") ||
// first.equalsIgnoreCase("executeLongOther")) {
// executeLongTaskOnOtherMember(args);
// } else if (first.equalsIgnoreCase("long") || first.equalsIgnoreCase("executeLong"))
// {
// executeLong(args);
} else if (first.equalsIgnoreCase("instances")) {
handleInstances(args);
} else if (first.equalsIgnoreCase("quit") || first.equalsIgnoreCase("exit")) {
System.exit(0);
} else {
println("type 'help' for help");
}
}
/**
* Discover WS device on the local network
*
* @return list of unique devices access strings which might be URLs in most cases
*/
public static Collection<String> discoverWsDevices() {
final Collection<String> addresses = new ConcurrentSkipListSet<>();
final CountDownLatch serverStarted = new CountDownLatch(1);
final CountDownLatch serverFinished = new CountDownLatch(1);
final Collection<InetAddress> addressList = new ArrayList<>();
try {
final Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
if (interfaces != null) {
while (interfaces.hasMoreElements()) {
NetworkInterface anInterface = interfaces.nextElement();
if (!anInterface.isLoopback()) {
final List<InterfaceAddress> interfaceAddresses = anInterface.getInterfaceAddresses();
for (InterfaceAddress address : interfaceAddresses) {
addressList.add(address.getAddress());
}
}
}
}
} catch (SocketException e) {
e.printStackTrace();
}
ExecutorService executorService = Executors.newCachedThreadPool();
for (final InetAddress address : addressList) {
Runnable runnable =
new Runnable() {
public void run() {
try {
final String uuid = UUID.randomUUID().toString();
final String probe =
WS_DISCOVERY_PROBE_MESSAGE.replaceAll(
"<wsa:MessageID>urn:uuid:.*</wsa:MessageID>",
"<wsa:MessageID>urn:uuid:" + uuid + "</wsa:MessageID>");
final int port = random.nextInt(20000) + 40000;
@SuppressWarnings("SocketOpenedButNotSafelyClosed")
final DatagramSocket server = new DatagramSocket(port, address);
new Thread() {
public void run() {
try {
final DatagramPacket packet = new DatagramPacket(new byte[4096], 4096);
server.setSoTimeout(WS_DISCOVERY_TIMEOUT);
long timerStarted = System.currentTimeMillis();
while (System.currentTimeMillis() - timerStarted < (WS_DISCOVERY_TIMEOUT)) {
serverStarted.countDown();
server.receive(packet);
final Collection<String> collection =
parseSoapResponseForUrls(
Arrays.copyOf(packet.getData(), packet.getLength()));
for (String key : collection) {
addresses.add(key);
}
}
} catch (SocketTimeoutException ignored) {
} catch (Exception e) {
e.printStackTrace();
} finally {
serverFinished.countDown();
server.close();
}
}
}.start();
try {
serverStarted.await(1000, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (address instanceof Inet4Address) {
server.send(
new DatagramPacket(
probe.getBytes(),
probe.length(),
InetAddress.getByName(WS_DISCOVERY_ADDRESS_IPv4),
WS_DISCOVERY_PORT));
} else {
server.send(
new DatagramPacket(
probe.getBytes(),
probe.length(),
InetAddress.getByName(WS_DISCOVERY_ADDRESS_IPv6),
WS_DISCOVERY_PORT));
}
} catch (Exception e) {
e.printStackTrace();
}
try {
serverFinished.await((WS_DISCOVERY_TIMEOUT), TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
executorService.submit(runnable);
}
try {
executorService.shutdown();
executorService.awaitTermination(WS_DISCOVERY_TIMEOUT + 2000, TimeUnit.MILLISECONDS);
} catch (InterruptedException ignored) {
}
return addresses;
}
public static void main(String[] args) {
// TODO main
OptionParser parser = new OptionParser();
parser.acceptsAll(Arrays.asList("h", "help"), "Show this help dialog.");
OptionSpec<String> serverOption =
parser
.acceptsAll(Arrays.asList("s", "server"), "Server to join.")
.withRequiredArg()
.describedAs("server-address[:port]");
OptionSpec<String> proxyOption =
parser
.acceptsAll(
Arrays.asList("P", "proxy"),
"SOCKS proxy to use. Ignored in presence of 'socks-proxy-list'.")
.withRequiredArg()
.describedAs("proxy-address");
OptionSpec<String> ownerOption =
parser
.acceptsAll(
Arrays.asList("o", "owner"), "Owner of the bot (username of in-game control).")
.withRequiredArg()
.describedAs("username");
OptionSpec<?> offlineOption =
parser.acceptsAll(
Arrays.asList("O", "offline"),
"Offline-mode. Ignores 'password' and 'account-list' (will "
+ "generate random usernames if 'username' is not supplied).");
OptionSpec<?> autoRejoinOption =
parser.acceptsAll(Arrays.asList("a", "auto-rejoin"), "Auto-rejoin a server on disconnect.");
OptionSpec<Integer> loginDelayOption =
parser
.acceptsAll(
Arrays.asList("d", "login-delay"),
"Delay between bot joins, in milliseconds. 5000 is "
+ "recommended if not using socks proxies.")
.withRequiredArg()
.describedAs("delay")
.ofType(Integer.class);
OptionSpec<Integer> botAmountOption =
parser
.acceptsAll(
Arrays.asList("b", "bot-amount"),
"Amount of bots to join. Must be <= amount of accounts.")
.withRequiredArg()
.describedAs("amount")
.ofType(Integer.class);
OptionSpec<String> accountListOption =
parser
.accepts(
"account-list",
"File containing a list of accounts, in username/email:password format.")
.withRequiredArg()
.describedAs("file");
OptionSpec<String> socksProxyListOption =
parser
.accepts(
"socks-proxy-list",
"File containing a list of SOCKS proxies, in address:port format.")
.withRequiredArg()
.describedAs("file");
OptionSpec<String> httpProxyListOption =
parser
.accepts(
"http-proxy-list",
"File containing a list of HTTP proxies, in address:port format.")
.withRequiredArg()
.describedAs("file");
OptionSet options;
try {
options = parser.parse(args);
} catch (OptionException exception) {
try {
parser.printHelpOn(System.out);
} catch (Exception exception1) {
exception1.printStackTrace();
}
return;
}
if (options.has("help")) {
printHelp(parser);
return;
}
final boolean offline = options.has(offlineOption);
final boolean autoRejoin = options.has(autoRejoinOption);
final List<String> accounts;
if (options.has(accountListOption)) {
accounts = loadAccounts(options.valueOf(accountListOption));
} else if (!offline) {
System.out.println("Option 'accounts' must be supplied in " + "absence of option 'offline'.");
printHelp(parser);
return;
} else accounts = null;
final String server;
if (!options.has(serverOption)) {
System.out.println("Option 'server' required.");
printHelp(parser);
return;
} else server = options.valueOf(serverOption);
final String owner;
if (!options.has(ownerOption)) {
System.out.println("Option 'owner' required.");
printHelp(parser);
return;
} else owner = options.valueOf(ownerOption);
final List<String> socksProxies;
if (options.has(socksProxyListOption))
socksProxies = loadProxies(options.valueOf(socksProxyListOption));
else socksProxies = null;
final boolean useProxy = socksProxies != null;
final List<String> httpProxies;
if (options.has(httpProxyListOption))
httpProxies = loadLoginProxies(options.valueOf(httpProxyListOption));
else if (!offline && accounts != null) {
System.out.println(
"Option 'http-proxy-list' required if " + "option 'account-list' is supplied.");
printHelp(parser);
return;
} else httpProxies = null;
final int loginDelay;
if (options.has(loginDelayOption)) loginDelay = options.valueOf(loginDelayOption);
else loginDelay = 0;
final int botAmount;
if (!options.has(botAmountOption)) {
System.out.println("Option 'bot-amount' required.");
printHelp(parser);
return;
} else botAmount = options.valueOf(botAmountOption);
initGui();
while (!sessions.get()) {
synchronized (sessions) {
try {
sessions.wait(5000);
} catch (InterruptedException exception) {
}
}
}
final Queue<Runnable> lockQueue = new ArrayDeque<Runnable>();
ExecutorService service = Executors.newFixedThreadPool(botAmount + (loginDelay > 0 ? 1 : 0));
final Object firstWait = new Object();
if (loginDelay > 0) {
service.execute(
new Runnable() {
@Override
public void run() {
synchronized (firstWait) {
try {
firstWait.wait();
} catch (InterruptedException exception) {
}
}
while (true) {
if (die) return;
while (slotsTaken.get() >= 2) {
synchronized (slotsTaken) {
try {
slotsTaken.wait(500);
} catch (InterruptedException exception) {
}
}
}
synchronized (lockQueue) {
if (lockQueue.size() > 0) {
Runnable thread = lockQueue.poll();
synchronized (thread) {
thread.notifyAll();
}
lockQueue.offer(thread);
} else continue;
}
try {
Thread.sleep(loginDelay);
} catch (InterruptedException exception) {
}
while (!sessions.get()) {
synchronized (sessions) {
try {
sessions.wait(5000);
} catch (InterruptedException exception) {
}
}
}
}
}
});
}
final List<String> accountsInUse = new ArrayList<String>();
for (int i = 0; i < botAmount; i++) {
final int botNumber = i;
Runnable runnable =
new Runnable() {
@Override
public void run() {
if (loginDelay > 0)
synchronized (lockQueue) {
lockQueue.add(this);
}
Random random = new Random();
if (!offline) {
boolean authenticated = false;
user:
while (true) {
if (authenticated) {
authenticated = false;
sessionCount.decrementAndGet();
}
Session session = null;
String loginProxy;
String account = accounts.get(random.nextInt(accounts.size()));
synchronized (accountsInUse) {
if (accountsInUse.size() == accounts.size()) System.exit(0);
while (accountsInUse.contains(account))
account = accounts.get(random.nextInt(accounts.size()));
accountsInUse.add(account);
}
String[] accountParts = account.split(":");
while (true) {
while (!sessions.get()) {
synchronized (sessions) {
try {
sessions.wait(5000);
} catch (InterruptedException exception) {
}
}
}
loginProxy = httpProxies.get(random.nextInt(httpProxies.size()));
try {
session = Util.retrieveSession(accountParts[0], accountParts[1], loginProxy);
// addAccount(session);
sessionCount.incrementAndGet();
authenticated = true;
break;
} catch (AuthenticationException exception) {
System.err.println("[Bot" + botNumber + "] " + exception);
if (!exception.getMessage().startsWith("Exception"))
// && !exception.getMessage().equals(
// "Too many failed logins"))
continue user;
}
}
System.out.println(
"["
+ session.getUsername()
+ "] Password: "******", Session ID: "
+ session.getSessionId());
while (!joins.get()) {
synchronized (joins) {
try {
joins.wait(5000);
} catch (InterruptedException exception) {
}
}
}
if (loginDelay > 0) {
synchronized (this) {
try {
synchronized (firstWait) {
firstWait.notifyAll();
}
wait();
} catch (InterruptedException exception) {
}
}
}
while (true) {
String proxy =
useProxy ? socksProxies.get(random.nextInt(socksProxies.size())) : null;
try {
new DarkBotMCSpambot(
DARK_BOT,
server,
session.getUsername(),
session.getPassword(),
session.getSessionId(),
null,
proxy,
owner);
if (die) break user;
else if (!autoRejoin) break;
} catch (Exception exception) {
exception.printStackTrace();
System.out.println(
"["
+ session.getUsername()
+ "] Error connecting: "
+ exception.getCause().toString());
}
}
System.out.println("[" + session.getUsername() + "] Account failed");
}
} else {
while (true) {
String proxy =
useProxy ? socksProxies.get(random.nextInt(socksProxies.size())) : null;
try {
String username = "";
if (accounts != null) {
username = accounts.get(random.nextInt(accounts.size())).split(":")[0];
synchronized (accountsInUse) {
while (accountsInUse.contains(username))
username = accounts.get(random.nextInt(accounts.size()));
accountsInUse.add(username);
}
} else
for (int i = 0; i < 10 + random.nextInt(6); i++)
username += alphas[random.nextInt(alphas.length)];
if (loginDelay > 0) {
synchronized (this) {
try {
synchronized (firstWait) {
firstWait.notifyAll();
}
wait();
} catch (InterruptedException exception) {
}
}
}
new DarkBotMCSpambot(DARK_BOT, server, username, "", "", null, proxy, owner);
if (die || !autoRejoin) break;
else continue;
} catch (Exception exception) {
System.out.println(
"[Bot" + botNumber + "] Error connecting: " + exception.toString());
}
}
}
}
};
service.execute(runnable);
}
service.shutdown();
while (!service.isTerminated()) {
try {
service.awaitTermination(9000, TimeUnit.DAYS);
} catch (InterruptedException exception) {
exception.printStackTrace();
}
}
System.exit(0);
}
/** 确定下载任务会调用线程池的shutdown()方法。之后不可以再添加新的下载任务 */
public static void ensure() {
ensured = true;
pool.shutdown();
}