*
News: SMF - Just Installed!


Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

unable to inventory vcenter from fsm 3555

Super Newbie Posts: 10 Karma: +0/-0 *
unable to inventory vcenter from fsm
« on: September 15, 2014, 03:57:21 PM »
Hello all,

we are trying to inventory our vcenter via the FSM and we are unable to discover it.  When we execute the discovery command the log reports the following:

September 15, 2014 3:54:47 PM EDT-Level:1-MEID:0--MSG: Job "System Discovery - xxx.xxx.xxx.xxx - September 15, 2014 3:54:47 PM EDT" activated.
September 15, 2014 3:54:47 PM EDT-Level:200-MEID:0--MSG: Subtask "System Discovery" activated.
September 15, 2014 3:54:47 PM EDT-Level:200-MEID:0--MSG: No clients to start.
September 15, 2014 3:54:47 PM EDT-Level:200-MEID:0--MSG: Subtask activation status changed to "Active".
September 15, 2014 3:54:47 PM EDT-Level:1-MEID:0--MSG: Job activation status changed to "Active".
September 15, 2014 3:54:47 PM EDT-Level:200-MEID:0--MSG: Subtask activation status changed to "Starting".
September 15, 2014 3:54:47 PM EDT-Level:150-MEID:0--MSG: ATKSRV623I Starting the discovery process for all resource types for IP address or range "xxx.xxx.xxx.xxx".
September 15, 2014 3:54:47 PM EDT-Level:200-MEID:0--MSG: Subtask activation status changed to "Active".
September 15, 2014 3:55:12 PM EDT-Level:150-MEID:0--MSG: ATKSRV616I Discovered 0 new systems.
September 15, 2014 3:55:12 PM EDT-Level:200-MEID:0--MSG: Subtask activation status changed to "Complete".
September 15, 2014 3:55:12 PM EDT-Level:1-MEID:0--MSG: Job activation status changed to "Complete".

The vcenter was originally discovered and so were all the hosts that it managed however it was accidentally deleted and now it cannot be re-discovered.

We are running FSM 1.3.2 and vCenter 5.5U1

Any assistance is appreciated.
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #1 on: September 16, 2014, 09:06:14 AM »
Emporio,

If you log into the FSM via SSH. Are you able to ping your vCenter server?
Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #2 on: September 17, 2014, 09:36:15 AM »
Hello,

yes we are able to ping vcenter by both IP and hostname
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #3 on: September 17, 2014, 06:33:06 PM »
Interesting. I have never seen that yet. Have you tried using the discover command? Maybe use the -v to see if it gives us any additional information? I would also verify that VMcontrol is running as I believe that is a requirement to accurately manage vCenter.

http://pic.dhe.ibm.com/infocenter/director/v6r1x/index.jsp?topic=/director.cli_6.1/fqm0_r_cli_discover.html
Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #4 on: September 30, 2014, 05:08:39 PM »
we are still unable to complete the discovery:

Here is the output:
USERID@FSM-MGMT:~> smcli discover -i xxx.xxx.xxx.xxx -v
Total discovery modules:
25

Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.agentmgr.AgentManagerDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.cim.CIMDiscoveryModule
Number of resource types:
4
Discovery module identifier:
com.ibm.sdnm.sdx.discovery.impl.SnmpDiscoveryModuleBase
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.hmc.HMCDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.imm2.StorageITEDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.imm.IMMDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.imm2.IMM2DiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.hypervisor.HypervisorDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.bcchassis.BCChassisDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.hw.bc2.discovery.BC2ChassisDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.bcchassis.BCChassisDotCommandDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.ssh.SSHDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.services.discovery.smis.SmisSlpDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.cas.CASDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.services.discovery.tpc.TPCDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.windows.WindowsDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.usmi.systems.discovery.cimlegacy.CIMLegacyDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.bmc.BMCDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.dpsm.discovery.bpc.BPCDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.tivoli.twg.tier.IPCDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.ivm.IVMDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.fsp.FSPDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systemz.zmgr.discovery.ZMgrDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.systems.discovery.hypervisor.IBMHypervisorDiscoveryModule
Number of resource types:
1
Discovery module identifier:
com.ibm.director.vcenter.VCenterDiscoveryModule
Discovery completion percentage 92%
Discovery completion percentage 96%
Discovery completion percentage 100%
Discovery result:
0Discovery identifier:
Discovery**FSM-MGMT##46Discovery completed:
100%
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #5 on: October 01, 2014, 09:20:44 AM »
Have you tried the following?

1) Reboot FSM -> retry discovery
2) Check firewall settings on the vCenter server (turn completely off for testing) FSM needs port 20000 to be open for communication with VMware.

Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #6 on: October 01, 2014, 11:59:59 AM »
A reboot of the FSM has not helped to resolve the issue. 
Also the windows firewall has been disabled to see if that would address the issue but we are still unable to complete the inventory.  I wonder if the system is thinking that the inventory is already completed and therefore it is not trying again, if that is the case i would like to know if there are some changes that can be made to either the vcenter server or the fsm to reset any possible connection that might be cached which is impeding the discovery from running again.
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #7 on: October 02, 2014, 10:20:55 AM »
Emporio,

The only thing I can offer at this point is to try the smcli rmsys -i xxx.xxx.xxx.xxx command. To see if for some reason the FSM still knows about the vCenter server. If it can't find it then I would open a ticket with IBM support on the issue. If it CAN find and remove it then try the rediscovery again. Something is definetely blocking it from being discovered. It is either in the FSM itself or something network related.

https://pic.dhe.ibm.com/infocenter/director/pubs/index.jsp?topic=%2Fcom.ibm.director.cli.helps.doc%2Ffqm0_r_cli_smcli.html
Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #8 on: October 02, 2014, 11:28:21 AM »
We attempted using the command you suggested but we are still unable to get the vcenter to be discovered.  I don't believe that the problem is with the FSM.  We have two vcenter (one for our main site and the other for our DR site).  At one time we had both discovered.  Now we can only manage the one at the DR site.  My feeling is that there is something stuck on the vcenter side which needs to be cleared in order to re-establish connection.

I am trying to follow the instructions on the article below as suggested by IBM support, but when I check, everything appears to be fine:

http://www-01.ibm.com/support/docview.wss?uid=nas71846c48b1342eea686257d2b0071b467

I am wondering if something with DCOM on the vcenter is misconfigured.
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #9 on: October 02, 2014, 03:31:41 PM »
Yes, it sounds like you are on the right track. Good information in the article as well. We will try to think about it on our end but keep us posted on your status!

Thanks!
Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #10 on: October 03, 2014, 03:43:17 PM »
Just for added information,  when we complete a scan of another system (windows server, non vcenter) we can see that on the event viewer - system logs there are a number of entries that are generated starting with the following:

DCOM  started the service TrustedInstaller with arguments "" in order to run the server:
{752073A1-23F2-4396-85F0-8FDB879ED0ED}

and followed by numerous entries like
The trisvc1702908167FSM-MGMT service entered the running state. (where the FSM-MGMT is the name of the our flex system manager server)
The trisvc1702908167FSM-MGMT service entered the stopped state. (where the FSM-MGMT is the name of the our flex system manager server)

These same entries do not occur when trying to discover the vcenter server at our primary site where we are experiencing problems with the inventory.

Also we confirmed that the same entries are also generated on the other vcenter server at our DR site that is inventoried properly.
Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #11 on: October 03, 2014, 04:27:16 PM »
A bit more information.  Upon starting a scan of any system (be it virtual or physical) there are no actual entries that occur in the eventvwr.  The entries I had mentioned in my earlier post are actually generated as soon as the credentials for said system are provided the FSM in order to access the remote server. At the time when the credentials are supplied, the system being inventoried executes a vbs in the c:\windows\temp directory which is then deleted within a few seconds, and completes the inventory of the server to report back to the fsm.

So the problem could be that the FSM server is trying to use credentials from the previous scan which need to be re-supplied, however since we cannot see the target in the list of discovered devices we cannot supply the new credentials

I am trying to see if there are other approaches we can take to force the discovery.
Administrator Sr. Member Posts: 309 Karma: +6/-0 *****
Re: unable to inventory vcenter from fsm
« Reply #12 on: October 09, 2014, 03:15:36 PM »
Now I'm confused. Is the vCenter server object there and you just can't inventory? Or are you trying to rediscover the vCenter server?

Super Newbie Posts: 10 Karma: +0/-0 *
Re: unable to inventory vcenter from fsm
« Reply #13 on: October 20, 2014, 02:08:04 PM »
The vcenter server was there but was accidentally deleted.  Now it is not re-inventorying properly.  If I search for the vcenter by either name or IP address nothing comes back to begin with, however if I start the inventory of it, I receive the message I posted earlier, and then when I search for resources by either name or IP addresses, I can see that some entries for it are created but not all.