Unable to get monitoring working in latest ngrinder

[John Warner]

Hello,

We've been struggling to get the monitor working with the latest version of ngrinder (3.3.1) and were wondering if you had any ideas.

It looks like the controller cannot communicate with the monitor's JMX bean over RMI, which is something I've confirmed by trying to connect using jmc. We've been all over the network settings and all ports are open, nothing is locked down and I've even tried it on an entirely separate network and on an internal network using VMWare and still it doesn't work.

I can see that the initial connection does make it through as I turned on logging within the monitor and I get this line output:

FINER: RMI TCP Connection(3)-10.43.196.192: [10.43.196.192: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]

10.43.196.192 is the IP address of the machine making the request, so it's definitely getting through, but then nothing happens and the request just hangs.

We've tried this with both Java 7 and Java 6 without any luck. The problem seems to be that it's disappearing into the RMI lookup method but I can't work out why.

Any ideas?

**EDIT** I should add that we have tried this on 3.3 (the latest release version) and then moved onto trying it with 3.3.1

--
John Warner

[junoyoon]

Currently, there is 1 second timeout logic when connecting monitor from controller not to be blocked in the monitor connection phase.

We observed it cause some problem when the target server is under stress. It delays connections and finally make the monitor connection failed.

We take this as a serious bug, we'll fix this in 3.3.1 soon. 

P.S We didn't release 3.3.1 yet. :-)

2014-04-07 22:15 GMT+09:00 John Warner [via ngrinder] <[hidden email]>:

Hello,

We've been struggling to get the monitor working with the latest version of ngrinder (3.3.1) and were wondering if you had any ideas.

It looks like the controller cannot communicate with the monitor's JMX bean over RMI, which is something I've confirmed by trying to connect using jmc. We've been all over the network settings and all ports are open, nothing is locked down and I've even tried it on an entirely separate network and on an internal network using VMWare and still it doesn't work.

I can see that the initial connection does make it through as I turned on logging within the monitor and I get this line output:

FINER: RMI TCP Connection(3)-10.43.196.192: [10.43.196.192: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]

10.43.196.192 is the IP address of the machine making the request, so it's definitely getting through, but then nothing happens and the request just hangs.

We've tried this with both Java 7 and Java 6 without any luck. The problem seems to be that it's disappearing into the RMI lookup method but I can't work out why.

Any ideas?

--
John Warner

---

If you reply to this email, your message will be added to the discussion below:

http://ngrinder.642.n7.nabble.com/Unable-to-get-monitoring-working-in-latest-ngrinder-tp1462.html

To start a new topic under ngrinder-user-en, email [hidden email]
To unsubscribe from ngrinder-user-en, click here.
NAML

[John Warner]

Hi,

Thanks for getting back to me. Yeah, I saw about the timeout, which is why I grabbed the source (3.3.1) and built everything myself with a much bigger timeout.

But that still didn't resolve the issue. It's almost like the RMI request isn't being handled correctly. It makes it into the RMI lookup but fails silently at that point. Unfortunately that's in code which I can't view in a debugger, so I can't see what it's up to.

It works when everything is on the same machine, but I see it takes a different approach to talking to the monitor at that point, so that makes sense. Have you seen it working remotely?

--
John Warner

[junoyoon]

Thanks for your feedback.

Yes. We already tested this before and I saw it worked before.

I have a question to you. Do your target have the multiple IPs?

2014-04-08 2:34 GMT+09:00 John Warner [via ngrinder] <[hidden email]>:

Hi,

Thanks for getting back to me. Yeah, I saw about the timeout, which is why I grabbed the source (3.3.1) and built everything myself with a much bigger timeout.

But that still didn't resolve the issue. It's almost like the RMI request isn't being handled correctly. It makes it into the RMI lookup but fails silently at that point. Unfortunately that's in code which I can't view in a debugger, so I can't see what it's up to.

It works when everything is on the same machine, but I see it takes a different approach to talking to the monitor at that point, so that makes sense. Have you seen it working remotely?

--
John Warner

---

If you reply to this email, your message will be added to the discussion below:

http://ngrinder.642.n7.nabble.com/Unable-to-get-monitoring-working-in-latest-ngrinder-tp1462p1464.html

To start a new topic under ngrinder-user-en, email [hidden email]
To unsubscribe from ngrinder-user-en, click here.
NAML

[junoyoon]

Could you open run_monitor.sh and fix the following line to have -Dcom.sun.management.jmxremote.authenticate=false and run it again?

java -server -cp "lib/*" -Dcom.sun.management.jmxremote.authenticate=false org.ngrinder.NGrinderAgentStarter --mode monitor --command run $@

2014-04-08 12:17 GMT+09:00 Juno Yoon <[hidden email]>:

Thanks for your feedback.

Yes. We already tested this before and I saw it worked before.

I have a question to you. Do your target have the multiple IPs?

2014-04-08 2:34 GMT+09:00 John Warner [via ngrinder] <[hidden email]>:

Hi,

Thanks for getting back to me. Yeah, I saw about the timeout, which is why I grabbed the source (3.3.1) and built everything myself with a much bigger timeout.

But that still didn't resolve the issue. It's almost like the RMI request isn't being handled correctly. It makes it into the RMI lookup but fails silently at that point. Unfortunately that's in code which I can't view in a debugger, so I can't see what it's up to.

It works when everything is on the same machine, but I see it takes a different approach to talking to the monitor at that point, so that makes sense. Have you seen it working remotely?

--
John Warner

---

If you reply to this email, your message will be added to the discussion below:

http://ngrinder.642.n7.nabble.com/Unable-to-get-monitoring-working-in-latest-ngrinder-tp1462p1464.html

To start a new topic under ngrinder-user-en, email [hidden email]
To unsubscribe from ngrinder-user-en, click here.
NAML

[John Warner]

Hi,

Thanks for the suggestion. While setting things up to try that, I found a problem with the configuration on our box. Essentially the hostname was being pointed to 127.0.1.1 instead of its external IP address.

That was preventing the connection being established correctly and caused all of our problems. All is well now.

Thanks for all of your help and sorry to have wasted your time.

--
John Warner

[junoyoon]

Oh. Thanks. However your case(hostname => 127.0.0.1) was what we thought that it's fixed.

Actually nGrinder JMX-Over-RMI implementation is supposed to be bound to all available IPs.

I'll follow up this issue and fix it soon.

2014-04-09 5:09 GMT+09:00 John Warner [via ngrinder] <[hidden email]>:

Hi,

Thanks for the suggestion. While setting things up to try that, I found a problem with the configuration on our box. Essentially the hostname was being pointed to 127.0.1.1 instead of its external IP address.

That was preventing the connection being established correctly and caused all of our problems. All is well now.

Thanks for all of your help and sorry to have wasted your time.

--
John Warner

---

If you reply to this email, your message will be added to the discussion below:

http://ngrinder.642.n7.nabble.com/Unable-to-get-monitoring-working-in-latest-ngrinder-tp1462p1471.html

To start a new topic under ngrinder-user-en, email [hidden email]
To unsubscribe from ngrinder-user-en, click here.
NAML

[junoyoon]

I think I found the root cause. nGrinder monitor only works properly when IP6 and IP4 are both supported. If IP6 is not supported in the target system. It tries to bind the default IP (IP bound to hostname)

Could you check your server has IP6 address(not including IP6 for loopback and tenneling interfaces)?

2014-04-09 22:44 GMT+09:00 Juno Yoon <[hidden email]>:

Oh. Thanks. However your case(hostname => 127.0.0.1) was what we thought that it's fixed.

Actually nGrinder JMX-Over-RMI implementation is supposed to be bound to all available IPs.

I'll follow up this issue and fix it soon.

2014-04-09 5:09 GMT+09:00 John Warner [via ngrinder] <[hidden email]>:

Hi,

Thanks for the suggestion. While setting things up to try that, I found a problem with the configuration on our box. Essentially the hostname was being pointed to 127.0.1.1 instead of its external IP address.

That was preventing the connection being established correctly and caused all of our problems. All is well now.

Thanks for all of your help and sorry to have wasted your time.

--
John Warner

---

If you reply to this email, your message will be added to the discussion below:

http://ngrinder.642.n7.nabble.com/Unable-to-get-monitoring-working-in-latest-ngrinder-tp1462p1471.html

To start a new topic under ngrinder-user-en, email [hidden email]
To unsubscribe from ngrinder-user-en, click here.
NAML

[John Warner]

That's good to know. We don't have IP6 enabled on the target system, so that's why it was trying to bind to the default IP.
--
John Warner