• Citrix StoreFront 2.x or 3.0
• NetScaler 10.5 and higher
• Citrix Receiver for Windows 4.x
• Citrix Receiver for Mac 11.8
• ​Web browser (Receiver for Web)
• ​Authentication configured on the NetScaler Appliance as outlined in CTX108876 - How to Configure LDAP Authentication on a NetScaler Appliance
• SSL Certificates configured for StoreFront Server and NetScaler Gateway Appliance. For additional information on setting up SSL certificates, refer to the following Citrix Documentation:
○ ​ Install and set up for StoreFront 2.6
○ Windows 2012 Server Certificates
○ To add an SSL binding to a site
○ Installing and Managing Certificates for NetScaler 10.
Pre-requisites
• NetScaler VPX, MPX, SDX appliance
• XenApp/XenDesktop
• Storefront Servers
• Communication ports open on the Firewall
○ http://support.citrix.com/article/CTX113250
○ http://support.citrix.com/article/CTX114355
○ https://support.citrix.com/servlet/KbServlet/download/2389-102-704421/CTX101810_28th_June_2013.pdf
When we plan to use the NetScaler Gateway for the users to get Storefront, there are a couple of process that work together in order to accomplish this task.
• Authentication (Who are you, and what are you allowed to get)
• Enumeration (Once authenticated, what apps do you have access to) [User able to get his Apps from Storefront through the Gateway]
• AppLauch (User try to get the App/Desktop from backend server)
Is really important to know this process in order to be able to troubleshoot any issue while working with the NetScaler Gateway and Storefront.
We will start this article describing the configuration steps and then with some troubleshooting advises when working with NetScaler Gateway deployments for Storefront.
StoreFront Configuration:
• Deploying Storefront server Pre-requisites:
○ https://docs.citrix.com/en-us/storefront/3/sf-plan.html
○ https://docs.citrix.com/en-us/storefront/3/sf-configure-server-group.html
• Configuration of Storefront for integration with NetScaler Gateway:
○ Pre-requisites
§ Valid Certificates Installed on Storefront Servers
□ http://support.citrix.com/article/CTX200292
□ http://support.citrix.com/content/dam/supportWS/kA260000000TWYbCAO/NSG_10.5_for_SF_2.6_and_XD_7.6.pdf
§ Valid Certificate Installed on the NetScaler
□ http://support.citrix.com/article/CTX109260
□ http://support.citrix.com/article/CTX136023
§ Load Balancing VIP configured on the NetScaler for Storefront Servers
□ https://docs.citrix.com/en-us/storefront/3/integrate-with-netscaler-and-netscaler-gateway/load-balancing-with-netscaler.html
§ DNS records configured for all the FQDN in use
Process:
○ Add/Remove Methods under Authentication
○ Check Pass-through from NetScaler Gateway for the authentication
○ Add NetScaler Gateway on the NetScaler Gateway tab
○ Add the NetScaler Gateway URL: FQDN that is going to be used to get to the NetScaler Gateway
○ Next
○ Add the Secure Ticket Authority Information
○ Add as many STA as required.
○ In newer Storefront version Load Balance STA is available. Screenshot from SF 3.5.
○ Create
○ Enable Remote Access to the Store
○ Done
NetScaler Gateway Configuration:
• In order to configure the NetScaler Gateway for Storefront configuration you will have to configure:
○ Authentication Policies (LADP, Radius)
○ Session Policies for Storefront
§ http://support.citrix.com/article/CTX139963
○ STA
• Using the configuration Wizard:
○ http://support.citrix.com/content/dam/supportWS/kA260000000TWYbCAO/NSG_10.5_for_SF_2.6_and_XD_7.6.pdf
○ https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/citrix-netscaler-and-citrix-xendesktop-7-deployment-guide.pdf
○ Select XenApp and XenDesktop Wizard
○ Get Started
○ Select between Web Interface/Storefront/WIonNS
○ Configure all the requested parameters. Please be advised that the NetScaler Gateway IP address is the one the users are going to be reaching to get to the Apps published through the NetScaler Gateway. You should include a DNS record for this IP and publish it internally/externally accordingly. The Gateway FQDN will be used to make the redirection from HTTP to HTTPs
○ Select the installed certificate on the NetScaler or install a new one for the NetScaler Gateway FQDN
○ Select the Primary and/or Secondary Authentication Methods
○ Fill all the needed fields for the Storefront Configuration
§ If load Balance have not been previously configured
§ If Load Balance was previously configured
○ Then Follow the wizard selecting the required compression methods and XenApp/XenDesktop Configuration
• Manually configuration of the Gateway
○ NetScaler Gateway > ADD
○ Configure the NetScaler Gateway IP
○ Bind the certificate
○ Select the authentication methods
○ Create and bind session policies as required.
§ http://support.citrix.com/article/CTX139963
Troubleshooting
When we plan to use the NetScaler Gateway for the users to get Storefront, there are a couple of process that work together in order to accomplish this task. So we have to understand which process is failing in order to start performing the right troubleshooting.
○ Authentication (Who are you, and what are you allowed to get)
○ Enumeration (Once authenticated, what apps do you have access to) [User able to get his Apps from Storefront through the Gateway]
○ AppLauch (User try to get the App/Desktop from backend server)
Users are unable to get the NetScaler Gateway Login page.
When the users are unable to get the NetScaler Gateway Login page it could be a L2-L4 problem between the end user computer and the NetScaler Gateway VIP. Some of the most common issues are described below.
§ Unable to resolve the NetScaler Gateway FQDN [Step 1]
□ DNS issue
® Common Error Message
This site can’t be reached
myapps.project.lab’s server DNS address could not be found.
Try running Windows Network Diagnostics.
DNS_PROBE_FINISHED_NXDOMAIN
§ You can resolve the FQDN but you are unable to get to the NetScaler Gateway VIP [Step 2]
□ Nat issue (if NAT is being used)
□ Firewall blocking the traffic to the NetScaler Gateway VIP
□ Routing issue (Packets unable to get to the NetScaler Gateway VIP)
So how do I know that the NetScaler Gateway VIP is listening and receiving traffic over 443 when I can't connect to the VIP? [Step 3]
User is able to get to the login page but have issues authenticating into the NetScaler Gateway. 'Incorrect credentials. Try again.' error
Common issues:
§ Wrong Username and Password
§ NetScaler unable to talk to the authentication servers
§ Invalid LDAP Bind Attempt
Fix: Use http://support.citrix.com/article/CTX114999 for troubleshooting.
For dual factor authentication use http://support.citrix.com/article/CTX125364.
Note: Only change in above article is that the credential Index under Client experience tab should be set to “Secondary” as LDAP is secondary for receiver access and we should be passing secondary credentials for SSO.
User is able to login successfully but….
○ Detect 'Cannot complete your request.' error after login
Fix: Use http://support.citrix.com/article/CTX207162 for troubleshooting
○ 'Http/1.1 Service Unavailable' error after login
Fix: [Step 4]
○ '403 - Forbidden: Access is denied.' error after login. Usually related to missing 'Web' at the end of the Web Interface Address field on the Published Application tab of the session profile for receiver for web poicy.
Fix: [Step 5]
If the user is able to login and enumerate apps there is only one process away to get his app: Applaunch. Applaunch is performed when the user click the desired application.
If Applaunch fails with the error: Cannot start app 'NAME_OF_THE_APP'
Fix: [Step 6]
No more than a number of users can lunch APPs.
Fix: [Step 7]
Not able to see apps or launch application through Mobile devices.
Fix: [Step 8]
a) First start by making sure config is correct on NETSCALER GATEWAY.Then make sure config is correct on backend WI or SF.
http://support.citrix.com/article/CTX139963
b) On WI you will be using PNAgent site or services site. Make sure it is set to Gateway Direct and NETSCALER GATEWAY FQDN is correct.
c) Try with different devices, iPhone, iPad, Android to see the behavior across all devices.
d) Make sure that his firewall is allowing port 80/443 from NETSCALER GATEWAY VIP to backend WI/SF server. If this port is not opened than apps will not be displayed.
e) What ports to be opened for WI/SF through NETSCALER GATEWAY check below article:
http://support.citrix.com/article/CTX114355
f) If you are using WI as backend to connect with Windows Receiver, then Authentication should be disabled on NetScaler Gateway. For mobile receivers it doesn't matter, but Windows and MAC receivers doesn't work if AUTH is enabled on NETSCALER GATEWAY.
g) To collect receiver logs:
http://support.citrix.com/article/CTX136380
Other errors:
○ EPA Scan for MAC and windows machines (EPA doesn’t work for mobile devices)
§ EPA is used for checking for a pre-defined policy on user computer before the user get the login prompt. If you detect EPA check failing, then disable the pre-authentication and check if the Login page fails as well. Check the below article for EPA scan on MAC: http://support.citrix.com/article/CTX134429.
§ For OPSWAT scans please refer below link: http://blogs.citrix.com/2014/09/26/how-to-securing-the-netscaler-gateway-using-opswat/
○ Opswat EPA scan failing:
§ First we need to confirm the type of scan customer is using. Check http://blogs.citrix.com/2014/09/26/how-to-securing-the-netscaler-gateway-using-opswat/
§ If you want to know if the expression is correct, we need to collect client side plugin logs. Make sure Client debug is set to debug on Gateway Vserver as mentioned in below link:
□ http://support.citrix.com/article/CTX138155
§ Add registry 'Enableepalogging' registry of type 'REG_DWORD' and value '1' at 'HKLMSoftwareCitrixSecure Access Client' 3. Reproduce EPA related issue, EPA log files (epahelper.txt and epaHelper_epa_plugin.txt) will now contain extra information about scan failure.
§ To collect logs on the Client computer, run C:Program FilesCitrixSecure Access ClientnsclientCollect.exe c:logs.zip.
Note: You can store and name the log file (c:logs.zip) anything you want as long as it has a .zip extension. A window as shown in the following screen shot appears for a few seconds to collect logs:
Extract the contents of the C:logs.zip file and review the log files.
○ For Web Interface on Windows getting error 401 unauthorized after authenticating through NetScaler Gateway using browser and URL showing agesso.aspx
§ Check Event viewer logs for any error messages like below:
□ INFO: ERROR: Event Log ID: 18001 A communication error occurred while attempting to contact the NetScaler Gateway authentication service at https://NETSCALER GATEWAYFQDN/CitrixAuthService/AuthService.asmx. Check that the authentication service is running
If we see this error make sure that:
Call back URL on WI site under 'Authentication Methods' on right side has correct NetScaler Gateway FQDN with complete path like this https://NETSCALERGATEWAYfqdn.com/CitrixAuthService/AuthService.asmx
WI should be able to resolve NetScaler Gateway FQDN and be able to get to the NetScaler Gateway VIP.
Workaround if not able to resolve: Make host file entry on WI server or change local DNS settings.
® Test with Internet Explorer on Web Interface server and browse to the NetScaler Gateway FQDN. We should get login page.
® Perform a telnet to NetScaler Gateway FQDN over port 443 and check if it's listening.
Check if XML service is running on correct port and it is not closed from Web Interface server under Server Farms.
In Session Profile on NetScaler make sure you have correct Domain specified for SSO.
If port is open or there is no firewall between WI and NS, it can be asymmetrical routing issue. To prove this, enable MAC based forwarding on NS and test. Once enabled, try to access NetScaler Gateway FQDN from Internet Explorer browser on Web Interface server and see if we get the login page. If this works, then definitely there is a routing issue.
○ Wi on ns issues: getting white screen after authenticating through NetScaler Gateway...url in browser says it’s stuck at AGESSO.JSP
Check:
§ Is NS able to resolve the NetScaler Gateway FQDN as the NetScaler Gateway VIP?
Workaround: Add DNS entry on the NS in NetScaler > Traffic Management > DNS > Records > Address Records
§ Check if cert is trusted in java keystore by using following command:
□ /var/wi/java_home/bin/keytool -list -keystore /var/wi/java_home/lib/security/cacerts
Check for the following certificates: Server cert, root cert and intermediate. If they are not linked, then manually run the following command to link the certs:
□ /var/wi/java_home/bin/keytool -import -trustcacerts -file /nsconfig/ssl/Veresign_Intermediate.crt -alias Verisign-Intermediate -keystore /var/wi/java_home/lib/security/cacerts
Note: MAKE CHANGES to following command according to your cert name. Alias name can be anything.
Check: http://support.citrix.com/article/CTX130435
NetScaler Gateway and Web Interface configuration article:
http://blogs.citrix.com/2012/04/10/netscaler-for-the-xendesktopxenapp-dummy/
Step 1:
Perform nslookup <Netscaler_Gateway_FQDN> from the client PC to the NetScaler Gateway FQDN
Or ping <Netscaler_Gateway_FQDN> for testing
Resolution:
Add DNS entry for the Netscaler Gateway FQDN on the public DNS
Workaround:
Modify the host file of the computer and manually add the entry located in C:WindowsSystem32driversetc.
Try to reach the site again:
Step 2:
From the Netscaler console, go to the shell and run the command:
nstcpdump.sh host <Netscaler_Gateway_VIP> and port 443
From the client computer perform a telnet to the Netscaler_Gateway_VIP over 443 and check on the Netscaler Console if you detect the packets hitting the VIP: telnet <Netscaler_Gateway_VIP> 443
Telnet Succeed:
Traffic Hitting the Netscaler Gateway VIP
Telnet Failed:
No Traffic Hitting the Netscaler Gateway VIP
When the telnet fails it could be related to a networking issue, Firewall rule blocking the traffic or NAT rule not working as expected.
Step 3
From the Netscaler console, go to the shell and run the command:
nstcpdump.sh host <Netscaler_Gateway_VIP> and port 443
Open a second SSH session to the Netscaler and whether a telnet Netscaler_Gateway_VIP 443 or curl -kv https://NETSCALER_GATEWAY_VIP from the shell
Port Opened:
Getting content from the VIP using curl to make an http request:
Note: the 302 error is expected due to that the login page is found on /vpn/index.html
Step 4:
'Http/1.1 Service Unavailable' error
This error means that the Netscaler is unable to talk to the backend Storefront servers, port 80/443 is being blocked from the SNIP to the backend server or IIS is down on the backend server.
Note: If we are load balancing Storefront servers we should test with only one Storefront service bound at the time to discard any possible failure associated only to one specific backed server.
Troubleshooting:
Perform a curl -kv < Web Interface Address field on the Published Application tab of the session profile> that is being applied to the user.
Failing:
Working:
Note: the HTTP/1.1 301 Moved Permanently is expected due to the real path is with '/' at the end of the address.
HTTP/1.1 200 OK
Sometimes curl would work due to the curl us being sourced from the NSIP instead of the SNIP. If curl works but you still get 'Http/1.1 Service Unavailable' error perform: telnet <STOREFRONT_FQDN used on Web Interface Address field on the Published Application tab of the session profile> 443/80 in order to check if the port is allowed from the SNIP or create a test service with HTTP/HTTPS monitor toward the backend Storefront servers
Failing
Working:
Monitor:
Step 5:
Go to VPN Virtual Server Session Policy Binding > Configure NetScaler Gateway Session Profile > Profile for Web and copy the path to get to the Receiver for Web from Storefront
Step 6:
Check Storefront server for any logs. Common errors:
§ All the configured Secure Ticket Authorities failed to respond to this XML transaction: https://10.125.248.203/scripts/ctxsta.dll.
§ SSL ERROR 43, 21 ETC OR PROTOCOL DRIVER ERROR OR SOCKET ERROR ON DESKTOP MACHINES.
Check if the STAs match on Storefront servers and Netscaler Gateway or if any STA is being marked as down on the Netscaler Gateway. If the STA is marked as down, perform a telnet <STA_IP> port from the Netscaler Shell. If the telnet fails, the port is being closed from the SNIP to the STA. {http://support.citrix.com/article/CTX114355}. You could create a test service toward the STA IP in order to check if the ports are being closed from the SNIP to the STA. If the service is marked as down, then the port is closed.
Check that Storefront is able to get to the STA by performing a telnet <STA_IP> port from the server shell.
Check that 1494/2598 ports are open from the SNIP toward the VDA. Perform a telnet <VDA_IP> 1494/2598 from the Netscaler Shell or create a test service toward the VDA IP in order to check if the ports are being closed from the SNIP to the VDA. If the service is marked as down, then the port is closed.
Step 7:
Check the license on Netscaler and the mode on the Netscaler Gateway Vserver:
On the Netscaler Gateway Vserver check if we are using SmartAccess mode or Basic mode (or in latest builds see if 'ICA Only' is checked or not). If using Smartaccess mode under Netscaler Gateway Vserver then Gateway User license will come into picture. If using Basic mode then ICA user license will come into picture.
Then go to System > License and check how much license we have for ICA (unlimited on the latest versions) and Gateway User License.
If the user claims he has 25 licenses but only 5 could launch apps, then go under Netscaler Gateway > Global Settings > Change Authentication settings and change the count for Max number of Users to whatever max license count we have.
Step 8:
First start by making sure config is correct on NETSCALER GATEWAY. Then make sure config is correct on backend WI or SF. Make sure the users are hitting the right policy when login the Netscaler Gateway and the policy is configured as below:
http://support.citrix.com/article/CTX139963
If using Wet Interface we are using PNAgent site or services site. Make sure it is set to Gateway Direct, the Netscaler Gateway FQDN is correct and the Web Interface server is able to resolver ad get to the Netscaler Gateway VIP (test with telnet and ping).
Try with different devices, iPhone, iPad, Android to see the behavior across all devices.
Make sure that the firewall is allowing port 80/443 from Netscaler Gateway VIP to backend Web_Interface/Storefront server. If this port is not opened than apps will not be displayed. In order to determine what ports need to be opened for Web_Interface/Storefront to work with Netscaler Gateway check below:
http://support.citrix.com/article/CTX114355
If we are using Web Interface backend server to connect with Windows Receiver, then Authentication should be disabled on Netscaler Gateway. For mobile receivers it doesn't matter, but Windows and MAC receivers doesn't work if AUTHENTICATION is enabled on Netscaler Gateway.
To collect receiver logs check below:
http://support.citrix.com/article/CTX136380
Some Useful Commands:
Check policy hits or what policy customer is hitting. From shell run below command to check policy hits:
nsconmsg -d current -g pol_hits
**************************************************
SSL Certificates Useful commands
**************************************************
Go to Shell type following commands:
cd /nsconfig/ssl
To convert .pfx to PEM(having cert and Private key) below command:
openssl pkcs12 -in certificate.pfx -out certificate.cer
Then you can extract private key and certificate individually from that file ad install it on Netscaler.
To convert cert to .pfx conversion:
openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt
Command line ssl cert install
add ssl certKey sslckey -cert server_cert.pem -key server_key.pem -password ssl
Command to create csr from command line....from command line runs the following:
create ssl certReq rajatnew1.csr -keyFile '/nsconfig/ssl/rajat.key' -countryName US -stateName 'New Jersey' -organizationName 'test test' -localityName test -commonName test -emailAddress rajat.mahajan@citrix.com
**********************
STA and what it does
**********************
Make sure STA defined on WI/SF are same as defined on NS. As it’s the WI/SF that decides which STA to choose and if that is not defined on NS then it can create issues in launching apps.
The STA Identifier should be different for different STA’s defined on Netscaler and can be seen in the Program Files(x86)CitrixSystem32CtxSta.config file, under the UID=STAFB06C8EF82 on backend WI or Xenapp server.
A user can launch a XenApp application with a XenDesktop server being used as STAs. It’s not mandatory that STAs need to be part of same XA/XD farm. It’s just a ticketing instance and isn’t farm specific in any manner.
To enable STA logging: http://support.citrix.com/article/CTX120589
STA FAQ: http://support.citrix.com/article/CTX101997
Reference Material:
http://support.citrix.com/content/dam/supportWS/kA260000000TWYbCAO/NSG_10.5_for_SF_2.6_and_XD_7.6.pdf
http://support.citrix.com/article/CTX139963
http://support.citrix.com/article/CTX114999
http://support.citrix.com/article/CTX114355
http://support.citrix.com/article/CTX113250
https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/citrix-netscaler-and-citrix-xendesktop-7-deployment-guide.pdf
http://support.citrix.com/article/CTX140153