Vuser Agent당 300명 테스트시 중단 관련 문의
Locked
 
|
This post was updated on .
안녕하세요^
Agent당 vUser 300이 이상 테스트시 테스트가 중단 되어 문의 드립니다. 컨트롤러 빨간색 공 메세지는 아래와 같습니다. The test is started Script error Stopped by error 컨트로는 메세지는 Script error 로 나오는데 스크립트(Groovy)는 웹에서 검증시 이상이 없었고 300명 이하 테스트는 정상적으로 수행되었습니다. 혹시 메모리 문제일까 해서 게시판의 여러글을 읽고 일단 버전 업그레이드(controller : 3.2.2 -> 3.2.3, Agent : 3.2.1-> 3.2.3)를 하고 agent의 JAVA option을 아래와 같이 추가 했습니다. # provide more agent java execution option if necessary. agent.javaopt=-Xms512m -Xmx1024m 현재 Agent는 아마존 인스턴스에 설치 되어 있고 메모리는 아래와 같습니다 . :~/ngrinder-core-3.2.3$ free total used free shared buffers cached Mem: 7629476 973900 6655576 0 155524 615944 -/+ buffers/cache: 202432 7427044 Swap: 0 0 0 free가 7.4G정도로 충분한거 같구요.. 이렇게 설정하고 테스트를 했는데 아래와 같이 Agent 에러가 납니다 . 2013-10-17 04:31:59,402 INFO net.grinder.AgentDaemon : Agent Daemon Agent conntected to port : 12001 is started. 2013-10-17 04:31:59,404 INFO agent controller : agent is started. waiting for agent controller signal 2013-10-17 04:31:59,416 INFO net.grinder.AgentDaemon : The Grinder 3.9.1 2013-10-17 04:31:59,464 INFO net.grinder.AgentDaemon : connected to console at 2013-10-17 04:31:59,465 INFO net.grinder.AgentDaemon : waiting for console signal 2013-10-17 04:32:00,345 INFO net.grinder.AgentDaemon : Clearing file store : : : 2013-10-17 04:32:01,839 INFO net.grinder.AgentDaemon : received a start 2013-10-17 04:32:02,199 INFO net.grinder.AgentDaemon : grinder properties {grinder.security=false, grinder.useConsole=true, grinder.consolePort=12001, grinder.duration=180000, ngrinder.etc.hosts= zonaws.com, grinder.runs=0, grinder.threads=38, grinder.agents=1, grinder.reportTimesToConsole=true, grinder.jvm=java, grinder.dcrinstrumentation=true, grinder.test.id=test_182, grinder.processIncrement=0, grinder.script=TestRunner.gro ovy, grinder.logProcessStreams=true, grinder.numberOfOldLogs=1, grinder.sleepTimeFactor=1, grinder.consoleHost=grinder.debug.singleprocess=false, grinder.logDirectory=/home/stpusr/.ngrinder_agent/log/test_182, grinder.sleepTimeVariation=0.2 , grinder.reportToConsole.interval=500, grinder.processes=9, grinder.user=admin, grinder.jvm.classpath=/home/stpusr/ngrinder-core-3.2.3/./lib/ngrinder-dns-3.2.3.jar:/home/stpusr/ngrinder-core-3.2.3/./lib/grinder-3.9.1-patch.jar:/home/stpusr/.ngrinder_agent /file-store/admin/current/, grinder.ignoreSampleCount=0} 2013-10-17 04:32:02,200 INFO net.grinder.AgentDaemon : jvm arguments -Djna.library.path=/home/stpusr/.ngrinder_agent/file-store/admin/current/lib -Dpython.path=/home/stpusr/.ngrinder_agent/file-store/admin/current/lib -Dpython.cachedir=/tmp/jython - Dngrinder.etc.hosts=Dsun.net.spi.nameservi ce.provider.1=dns,LocalManagedDns -Xms512m -Xmx1024m -Xms386m -Xmx386m -XX:PermSize=128m -XX:MaxPermSize=128m 2013-10-17 04:32:02,218 INFO net.grinder.AgentDaemon : Dsun.net.spi.nameservice.provider.1=dns,LocalManagedDns' '-Xms512m' '-Xmx1024m' '-Xms386m' '-Xmx386m' '-XX:PermSize=128m' '-XX:MaxPermSize=128m' -classpath '../../../../ngrinde r-core-3.2.3/lib/ngrinder-dns-3.2.3.jar:../../../../ngrinder-core-3.2.3/lib/grinder-3.9.1-patch.jar:.:../../../../ngrinder-core-3.2.3/lib/grinder-dcr-agent-3.9.1.jar:../../../../ngrinder-core-3.2.3/lib/ngrinder-groovy-3.2.3.jar:../../../../ngrinder-core-3. 2.3/lib/commons-io-2.0.1.jar:../../../../ngrinder-core-3.2.3/lib/dnsjava-2.1.1.jar:../../../../ngrinder-core-3.2.3/lib/hamcrest-all-1.1.jar:../../../../ngrinder-core-3.2.3/lib/ngrinder-dns-3.2.3.jar:../../../../ngrinder-core-3.2.3/lib/groovy-all-2.0.6.jar: ../../../../ngrinder-core-3.2.3/lib/json-20090211.jar:../../../../ngrinder-core-3.2.3/lib/logback-core-1.0.0.jar:../../../../ngrinder-core-3.2.3/lib/asm-3.2.jar:../../../../ngrinder-core-3.2.3/lib/grinder-core-3.9.1.jar:../../../../ngrinder-core-3.2.3/lib/ grinder-httpclient-3.9.1.jar:../../../../ngrinder-core-3.2.3/lib/slf4j-api-1.6.4.jar:../../../../ngrinder-core-3.2.3/lib/jna-3.4.0.jar:../../../../ngrinder-core-3.2.3/lib/junit-dep-4.8.2.jar:../../../../ngrinder-core-3.2.3/lib/picocontainer-2.13.6.jar:../. ./../../ngrinder-core-3.2.3/lib/grinder-3.9.1.jar:../../../../ngrinder-core-3.2.3/lib/commons-lang-2.6.jar:../../../../ngrinder-core-3.2.3/lib/logback-classic-1.0.0.jar:../../../../ngrinder-core-3.2.3/lib/grinder-http-3.9.1.jar' net.grinder.engine.process. WorkerProcessEntryPoint 2013-10-17 04:32:02,238 INFO net.grinder.AgentDaemon : worker ip 2013-10-17 04:32:02,247 INFO net.grinder.AgentDaemon : worker ip 2013-10-17 04:32:02,266 INFO net.grinder.AgentDaemon : worker ip 2013-10-17 04:32:02,286 INFO net.grinder.AgentDaemon : worker ip 2013-10-17 04:32:02,326 INFO net.grinder.AgentDaemon : worker ip- 2013-10-17 04:32:02,362 INFO net.grinder.AgentDaemon : worker 2013-10-17 04:32:02,392 INFO net.grinder.AgentDaemon : 2013-10-17 04:32:02,438 2013-10-17 04:32:43,643 INFO net.grinder.AgentDaemon : all workers are finished 2013-10-17 04:32:43,647 INFO net.grinder.AgentDaemon : communication shut down 2013-10-17 04:32:43,657 INFO net.grinder.AgentDaemon : finished 로그 상에는 큰 특이사항이 보이지 않는같은데요 테스트 수행하면서 Agent접속해 있으면 아래와 같은 로그가 보입니다. 2013-10-17 02:30:19,148 ERROR worker-bootstrap: Error running worker process net.grinder.communication.CommunicationException: Exception whilst sending message at net.grinder.communication.AbstractSender.send(AbstractSender.java:57) ~[grinder-core-3.9.1.jar:na] at net.grinder.communication.QueuedSenderDecorator.flush(QueuedSenderDecorator.java:60) ~[grinder-core-3.9.1.jar:na] at net.grinder.engine.process.GrinderProcess.sendStatusMessage(GrinderProcess.java:568) ~[grinder-3.9.1-patch.jar:na] at net.grinder.engine.process.GrinderProcess.run(GrinderProcess.java:378) ~[grinder-3.9.1-patch.jar:na] at net.grinder.engine.process.WorkerProcessEntryPoint.run(WorkerProcessEntryPoint.java:87) [grinder-core-3.9.1.jar:na] at net.grinder.engine.process.WorkerProcessEntryPoint.main(WorkerProcessEntryPoint.java:60) [grinder-core-3.9.1.jar:na] Caused by: java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_39] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) ~[na:1.6.0_39] at java.net.SocketOutputStream.write(SocketOutputStream.java:136) ~[na:1.6.0_39] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) ~[na:1.6.0_39] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) ~[na:1.6.0_39] at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1793) ~[na:1.6.0_39] at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:699) ~[na:1.6.0_39] at net.grinder.communication.AbstractSender.writeMessageToStream(AbstractSender.java:90) ~[grinder-core-3.9.1.jar:na] at net.grinder.communication.StreamSender.writeMessage(StreamSender.java:70) ~[grinder-core-3.9.1.jar:na] at net.grinder.communication.AbstractSender.send(AbstractSender.java:53) ~[grinder-core-3.9.1.jar:na] ... 5 common frames omitted 참고로 컨트롤러에는 아래와 같이 로그가 찍힙니다. 2013-10-17 04:28:25,264 INFO AgentManager.java:394 : 1 agents are starting for user admin 2013-10-17 04:28:25,264 INFO AgentManager.java:396 : - Agent nGrinderAgent01 2013-10-17 04:28:27,473 INFO SingleConsole.java:1304 : Sampling is started 2013-10-17 04:29:14,249 ERROR PerfTestRunnable.java:404 : Abnormal test 181 due to SCRIPT_ERROR 2013-10-17 04:29:15,650 ERROR PerfTestRunnable.java:478 : Terminate 181 2013-10-17 04:29:15,653 INFO SingleConsole.java:1317 : Sampling is stopped 2013-10-17 04:29:15,667 INFO MonitorCollectorListener.java:92 : remove monitors on [org.ngrinder.model.AgentInfo@48805ebb agentIdentity=<null>,hostName=,status=<null>,syst emStat=<null>,region=<null>,number=0,approved=false,id=<null>]] 2013-10-17 04:29:15,669 INFO PerfTestService.java:1176 : Total Statistics for test 181 is {} 2013-10-17 04:31:52,303 ERROR PerfTestService.java:850 : Report data for TPS in 181 does not exist. 2013-10-17 04:31:56,802 INFO ConsoleFoundationEx.java:169 : console 10.134.157.210:12001 has been started 2013-10-17 04:31:57,088 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/ for test 182 2013-10-17 04:31:57,103 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/ for test 182 2013-10-17 04:31:57,117 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/com/sec/stp/config for test 182 2013-10-17 04:31:57,134 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/XXXXXXX for test 182 2013-10-17 04:31:57,150 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/XXXXXXX for test 182 2013-10-17 04:31:57,167 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182XXXXXXX for test 182 2013-10-17 04:31:57,185 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182XXXXXXX for test 182 2013-10-17 04:31:57,201 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/XXXXXXX for test 182 2013-10-17 04:31:57,217 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/distXXXXXXXfor test 182 2013-10-17 04:31:57,233 INFO ScriptHandler.java:160 : XXXXXXX is being written in 2013-10-17 04:31:57,265 INFO ScriptHandler.java:160 : XXXXXXX is being written in /home/stpusr/.ngrinder/perftest/0_999/182/dist/com/sec/stp/test for test 182 2013-10-17 04:31:59,317 INFO GroovyMavenProjectScriptHandler.java:157 : Dependencies in GroovyTest/pom.xml is copied into /home/stpusr/.ngrinder/perftest/0_999/182/dist/lib folder 2013-10-17 04:31:59,318 INFO PerfTestService.java:786 201:31:59,339 INFO PerfTestService.java:754 : Grinder Properties : {grinder.threads=38, grinder.numberOfOldLogs=1, grinder.jvm com, grinder.user=admin, grinder.reportTimesToConsole=true, grinder.debug.singleprocess=false, grinder.processes=9, grinder.ignoreSampleCount=0, grinder.reportToConsole.interval=500, grinder.agents=1, grinder.dcrinstrumentation=true, grinder.logProcessStre ams=true} 2013-10-17 04:31:59,358 INFO AgentManager.java:394 : 1 agents are starting for user admin 2013-10-17 04:31:59,358 INFO AgentManager.java:396 : - Agent nGrinderAgent01 2013-10-17 04:32:01,855 INFO SingleConsole.java:1304 : Sampling is started 2013-10-17 04:32:46,819 ERROR PerfTestRunnable.java:404 : Abnormal test 182 due to SCRIPT_ERROR 2013-10-17 04:32:49,169 ERROR PerfTestRunnable.java:478 : Terminate 182 2013-10-17 04:32:49,171 INFO SingleConsole.java:1317 : Sampling is stopped 2013-10-17 04:32:49,172 INFO MonitorCollectorListener.java:92 : remove monitors on [org.ngrinder.model.AgentInfo@53635a08[ip=XXXXX,port=XXXXX,agentIdentity=<null>,hostName=,status=<null>,syst emStat=<null>,region=<null>,number=0,approved=false,id=<null>]] 2013-10-17 04:32:49,175 INFO PerfTestService.java:1176 : Total Statistics for test 182 is {} 2013-10-17 04:32:53,673 ERROR PerfTestService.java:850 : Report data for TPS in 182 does not exist. 2013-10-17 04:48:49,302 ERROR PerfTestService.java:850 : Report data for TPS in 182 does not exist. 어떤 문제인지 확인 부탁드립니다. |
Administrator
|
ulimit 의 문제인듯 보입니다. 현재 진행중이신 테스트가 HTTP 테스트 인가요?
-----Original Message----- If you reply to this email, your message will be added to the discussion below:
http://ngrinder.642.n7.nabble.com/Vuser-Agent-300-tp994.html
To start a new topic under ngrinder-user-kr, email [hidden email]
To unsubscribe from ngrinder-user-kr, click here. NAML |
|
|
네~ http request(post방식)으로 보내는 테스트입니다.
ulimit 문제는 어떻게 해결할 수 있나요? |
|
Administrator
|
http 의 경우에는 ulimit 의 문제라고 단정할수도 없어 보입니다.
300 vuser 를 잡을경우 TPS는 어느정도인지 그리고 어떻게 스크립트를 작성하셨는지 알려주세요. 스크립트는 중요 정보는 제거한 뒤(URL이나.. 파라미터는 제거..) 제 개인 메일로 보내주세요. |
|
Administrator
|
메일로 확인 한 결과 두가지 문제로 밝혀 졌습니다. 이 건에 대해 정리 차원에서 게시판에 글을 남깁니다.
1. 매 반복마다 암호화 라이브러리 초기화를 하였다. 자바의 Cipher 라이브러리는 초기화때 많은 메모리가 필요합니다. 그런데 매번 반복 수행할 때마다 Cipher 를 초기화 함으로써, 메모리가 급격히 소진되어, 테스트가 중단 되었습니다. 2. 단순 스크립트 오류.. 프로세스에 따라 다른 처리를 하도록 메소드를 별도로 구성하였는데, Vuser 를 많이 사용하면서, 프로세스 수가 늘어나고, 이에 따라 호출 가능한 메소드 범위를 벗어나서 생긴 문제. 결국 스크립트 문제였으나, ngrinder 에서 이런 특수한 상황에 대헤 로그를 정확히 출력하지 않아 문제를 해결하는데 사용자가 시간이 걸렸습니다. 이 문제는 일단 ngrinder 개발쪽에서 재현한 뒤에 3.3 릴리즈시에 반영하도록 하겠습니다. |
| Free forum by Nabble | Edit this page |