Oracle ATG – Endeca integration: Baseline Index process troubleshooting

This article contains troubleshooting steps for when an Endeca Baseline Index, as manually launched via the ATG Dyn Admin | ProductCatalogSimpleIndexingAdmin component, doesn't complete successfully.

All of this information is from my own experience and troubleshooting in trying to get an ATG-Endeca integration for a new ATG/Endeca application wired up and working successfully. (Which I did eventually accomplish, but not until after working through a fair number of unexpected issues, as should be apparent from the list below!) 

So, disclaimer: The suggested resolutions may or may not be “best practice” or otherwise work well for your own application. I’m just posting this information in hopes that it may be helpful; I certainly wished at points that there was more ATG-Endeca integration help available on the web at times, while I myself was initially troubleshooting my way through each one of these issues!

---

Q: The ProductCatalogSimpleIndexingAdmin doesn't exist in Dyn Admin. Searching for it returns 0 results. Browsing for it shows that the /atg/endeca subtree doesn't exist.

1. The "ATG-Endeca integration" feature might not be installed in ATG. This can be done with the atgdir/home/bin/sim.sh utility.

2. The application startup might not have included the required Nucleus modules for the Endeca indexing integration. In my situation, I found that I needed to add the following Make sure the following Nucleus components are included with the application server startup – these may or may not be the specific set of modules that are appropriate for your ATG application:

• DAF.Endeca.Index
• DCS.Endeca.Index

---

Q: In Dyn Admin, the ProductCatalogSimpleIndexingAdmin component does come up, but the special UI that normally appears at the top of that component's Dyn Admin page, that allows a Baseline Index to be launched and shows the status of the process, doesn't appear at all.

Try inspecting the application startup log for errors and warnings.  I saw this happen when my product-sku-output-config.xml file contained an extra “-->” token; this was reported as a parse error in the application startup logs.

---

Q: Errors about missing tables appear in the log, including table srch_site_content, both at server startup and when trying to run Baseline Index.

I ended up needing to manually run some ddl files in my local dev environment to resolve this. I ran:

On core:

• search_site_ddl.sql

On cata and catb:

• versioned_search_site_ddl.sql

These scripts are packaged with ATG. Look for them in one of the descendant directories of the ATG_HOME directory.

---

Q: Under JBoss, trying to run Baseline Index via ProductCatalogSimpleIndexingAdmin fails. Exception appears in the log: "atg.repository.search.indexing.IndexingException: java.lang.NoSuchMethodError: org.apache.cxf.transport.AbstractTransportFactory.<init>(Ljava/util/List;)"

This is apparently due to a version conflict between the version of CXF that comes packaged with JBoss, and the version of CXF that ATG assumes will be available to it.

I spent some time looking into how to disable or otherwise not use the CXF version bundled with JBoss (and use ATG's version of CXF instead), but never found a solution. I ended up switching my local environment to use WebLogic instead or JBoss as a workaround.

Here's a post on the Oracle forums with this same problem (and no solution, as of 5/2013): https://forums.oracle.com/forums/thread.jspa?messageID=10622176

---

Q: The second portion of the Baseline Index process (RepositoryExport) fails immediately.

Possible additional symptom in log: /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin ---java.lang.RuntimeException: org.apache.commons.httpclient.ConnectTimeoutException: The host did not accept the connection within timeout of 30000 ms

There might be a problem with the connectivity between ATG and Endeca CAS – possibly a configuration issue or networking issue. Check:

1. In the DimensionDocumentSubmitter, SchemaDocumentSubmitter, and DataDocumentSubmitter components, are the CASHostName and CASPort properties set properly? If not, make the changes in the corresponding .properties files for each of those components (in ATG 10.1.1), or in the /atg/endeca/index/IndexingApplicationConfiguration/ component (in ATG 10.1.2).

2. Try pinging the target Endeca CAS server from the command line on the ATG machine to verify that it is accessible on the network. 

3. Verify that DNS resolution isn’t misconfigured such that the CAS hostname is resolving to an IP address of some server other than the actual CAS server.

---

Q: The second portion of the Baseline Index process (RepositoryExport) fails immediately. The log includes "Response-Code: 500"

The "500" here appears to be an HTTP 500 ("Internal server error") being returned from Endeca CAS from the web service call. I've seen this happen several times when attempting to run a Baseline Update after a previous Baseline Update failed midway through, and when the indexing process is being run from the ATG CA server.

The only workaround I know of at this point is to bounce the CAS server. Quick instructions to bounce CAS (in my Endeca environment; specifics for your environment may vary):

1. ssh to the Endeca indexing server.
2. Change directory to endeca/CAS/version number/bin
3. ./cas-service-shutdown.sh
4. nohup ./cas-service.sh &

---

Q: The second portion of the Baseline Index process (RepositoryExport) hangs for a while, and then fails with no records successfully processed. An error related to network connectivity appears in the ATG server log.

1. Is CAS running? Log on to the Endeca indexing server and run:

ps -ef | grep cas

If there's no result, then start CAS.

2. Alternatively, this might indicate a DNS issue. Make sure that when you ping the Endeca indexing server from your ATG server, the correct IP address is being returned.

---

Q: java.io.IOException: http://[endeca host]/[endeca app name]en_en_dimvals/?wsdl returned response code 404

This appears to be caused by an incorrect defaultRecordStoreName and/or endecaBaseApplicationName entry in the DataDocumentSubmitter, DimensionDocumentSubmitter, and/or SchemaDocumentSubmitter components (ATG 10.1.1) or in the IndexingApplicationConfiguration component (ATG 10.1.2). Check on those values for those components in Dyn Admin and/or in the application’s .properties files.

One specific thing to check: If the language code (e.g. “en”) is being specified in the endecaBaseApplicationName property, this might result in a defaultRecordStoreName value with too many language code values, e.g. “MyAppenen_en_dimvals”, which isn't correct – too many "en".

---

Q: Exception in thread "index-/atg/endeca/index/commerce/ProductCatalogSimpleIndexingAdmin" java.lang.NullPointerException: Property value cannot be null (dimval.display_name)

This apparently indicates that one or more of the item-descriptor tags in customCatalog.xml (in the ATG project) are missing display-name attributes. (Note that some item-descriptor tags – particularly, the OOTB ones – inherit a display-name via the xml-combine="append" attribute value.)

Another symptom of this: In Dyn Admin, RepositoryTypeDimensionExporter component, under "show XML", one of the dimval.display_name entries in the XML shown has an empty PVAL.

---

Q: The ProductCatalogOutputConfig step reports success, but shows 0 records processed. The server log doesn't show any warnings.

Additional symptom: The Dyn Admin | Commerce Admin | Catalog Verification reports thousands of warnings like: "/atg/commerce/catalog/custom/CatalogVerificationService Product 434318 is not in any catalog. Cannot verify info objects."

Additional symptom: When looking the properties for any particular product via the ProductCatalog component in dyn admin, several properties are missing: ancestorCategories, ancestorCategoryIds, siteIds, parentCategoriesForCatalog, catalogs, computedCatalogs.

To fix:

• In Dyn Admin | Commerce Admin, run Catalog Update and then Basic Maintenance. Then, run Catalog Verification; it should report 0 errors, 0 warnings.
• If an error occurs during Basic Maintenance, it might be caused by an database deadlock. (You can look at the application log file to verify whether or not this is the case.) 

---

Q: ProductCatalogOutputConfig step fails after around 200 records processed. Application log shows: SQLException: Internal error: Cannot obtain XAConnection weblogic.common.resourcepool.ResourceLimitException: No resources currently available in pool

This apparently indicated that ATG was trying to use more connections than were available in WebLogic. I’m not sure what caused this when I saw it.  I ended up just retrying the baseline index (without changing any settings or bouncing the server), and it worked the second time.

---

Q: java.lang.OutOfMemoryError occurs after processing some, but not all, of the product records.

I worked around this by increasing the Java max heap size (for the ATG application) to 3G, and the MaxPermSize to 384m.

---

Q: Silent failure on the EndecaIndexing step (the last step). Nothing appears in the ATG application log after the RepositoryExport steps completed. The Dyn Admin GUI just reports "COMPLETE (Failed)".

1. This may indicate that the forge process failed on the Endeca side.  Try looking at the logs on the Endeca indexing server for details about the failure.

2. Verify that EAC is running on the Endeca indexing server.

3. Verify that in the EndecaScriptService (ATG 10.1.1) or ApplicationConfiguration (ATG 10.1.2) component, the endecaBaseApplicationName (10.1.1) or baseApplicationName (10.1.2) property has the correct value.

---

Q: Second portion of baseline index (RepositoryExport) fails midway through. Error in log: "/atg/search/repository/BulkLoader — atg.repository.search.indexing.IndexingException: com.endeca.itl.recordstore.ConcurrentWriteException: Write in progress with generation 12"

I found that a workaround for this was to restart CAS on the Endeca indexing server, then retry the baseline import from the ATG side.

Update 8/19/2013: Apparently this issue can be caused by an update lock not being released when a previous baseline index operation was manually canceled. See article "Cancelling the Baseline Indexing Job from SimpleIndexingAdmin does not Release Update Lock (Doc ID 1576472.1)" on the Oracle Support site for more information.

Workaround: Mac Mini outputs no video signal with DisplayPort-DVI adapter (and some amazing customer service)

Executive Summary / tl;dr

• A Mac Mini might produce no video output when connected to a ViewSonic or Dell monitor via a DVI-DisplayPort adapter.  Try using a DVI-MiniDVI adapter instead.
• For incredible customer service, buy Mac stuff from eBay seller dvicetech.

Although I’m a longtime Windows user when it comes to desktop PCs, over the past couple of years I have developed a strong liking of Apple’s iPod Touch as my “always-in-my-pocket” device of choice (as an inexpensive smartphone alternative).  With 3 young kids in the house, my free gaming time is increasingly spent on the iPod, as that platform is conducive to 10-15 minute play sessions; however, the particular game I’d like to play is not currently on the App Store, so being a programmer, I decided to write it myself!  (Hopefully, more to come on that topic in a future post on this blog!)

Being brand new to iOS development, I did some research, and ended up with the conclusion that unfortunately, using my existing Windows PC would not be a very good idea; Apple really wants you to be using a Mac for iOS development.  Being on a fairly constrained budget for discretionary purchases – again, see “3 young kids” above! – I opted for the “Mac Mini from eBay” option, as suggested on StackOverflow.

After examining the available options, and (as always with this type of purchase) looking for a sweet spot in the trade-off between price and performance, I opted for a so-called “Early 2009” model Mac Mini, with 2.0 GHz dual-core processor, 4 GB RAM, 500 GB HD, and the as-of-now latest OS, OS X 10.8 Mountain Lion, from eBay seller dvicetech.  This Mac Mini cost $444.50, including shipping; using my existing keyboard, mouse, and monitor, that was the total cost for the Mac.  (Definitely a more economical option than going with a new MacBook, which seem go in the $1100 – 1700 range.)

When my Mac Mini arrived, I hooked it up, connecting a USB mouse, USB keyboard, power, and my existing DVI monitor via a DisplayPort-DVI adapter included by the seller.  I turned the machine on, at which point the “power” light on the front of the Mac Mini turned on, I heard the Apple startup chime from the Mac’s built-in speakers, and… nothing at all appeared on the screen.  The monitor’s status indicator light remained yellow, as though no input source were connected.

This monitor, a ViewSonic, did work fine with my Windows PC; I had just unhooked it from that PC in order to use it with the Mac.

I checked the standard set of things that I could think of to check:

• Monitor powered on?  Yes, the yellow status indicator light was on.
• Mac powered on?  Yes, the power indicator LED was on and I heard the startup chime.  I could also toggle Caps Lock on and off via the keyboard and see the Caps Lock indicator light on the keyboard turn on and off.
• Video cables connected tightly to Mac and to monitor?  Yes.
• Monitor configured for DVI input?  Yes, verified.  I’d also been using this monitor with my Windows PC via DVI with no problems.
• DVI cable okay?  I swapped it out for a different DVI cable that I had on hand, but got the same results.

At this point, I disconnected the ViewSonic monitor, connected my other on-hand monitor (a Dell), and rebooted the Mac Mini.  This also produced the same results: Mac Mini appeared to power up okay, but no display output whatsoever.

So at this point, I concluded that the problem was either with the DisplayPort-DVI adapter, or with the Mac Mini itself.  I Googled using some general terms (“mac mini no video”), and came up with a Apple support article for this specific problem, which had several suggested solutions.  Unfortunately, when I tried the solutions, including several new-to-me troubleshooting actions like “reset the parameter RAM” and “reset the System Management Controller (SMC)”, none of them resolved the problem.  (Also, I was unable to try at least one of the listed solutions, which was to put the Mac to sleep by pressing Option+Command+Eject; the PC keyboard I was using with the Mac Mini in accordance with the Mac Mini’s advertised B.Y.O.D.K.M. feature didn’t have any of those keys!)

Having run out of troubleshooting steps to try, I contacted the eBay seller, dvicetech, through eBay’s nice dispute resolution site feature and described the problem, in hopes of being able to either get help in resolving the problem somehow (although my hopes for that were limited at this point), or else exchange the Mac Mini for another one, or get a refund. 

The seller, dvicetech, immediately suggested shipping me out a replacement DisplayPort-DVI adapter, at his own expense. This really pleased me, since combined with dvicetech taking on this extra cost, it seemed like a very reasonable next step to try – hopefully the problem would turn out to be with the adapter, not the Mac Mini itself. 

Unfortunately, when the new DisplayPort-DVI adapter arrived and I hooked it up, it didn’t resolve the issue. 

As a last-ditch troubleshooting attempt before I contacted dvicetech again with the bad news and to ask for either a replacement or a refund – I was leaning toward the latter, as at least then I’d have the option of spending an additional $200 or so on a new Mac Mini and engaging Apple Support if something were to go wrong with that – I brought the Mac Mini and the DisplayPort-DVI adapter into the office to try it with the monitors there, on the wild theory that there was some problem with both the ViewSonic and Dell monitors I had at home where those monitors worked fine with Windows 7, but didn’t work with the Mac. 

To my shock, this worked!  When I connected the Mac Mini to the Samsung DVI monitor that I had at work – using the same Mac Mini and DVI-DisplayPort adapter that had previously failed to produce any video signal on either the ViewSonic or Dell monitors that I had at home – I did get signal on the Samsung monitor.  I was able to watch the Mac Mini boot to the MacOS desktop with no issues.

I contacted dvicetech with this news.  At this point, dvicetech wowed me further with his great service by shipping out not one but two additional display adapters to try: A miniDVI-DVI adapter, and a miniDVI-VGA adapter – along with a postage-paid return envelope for whatever display adapters I didn’t end up using!  I hadn’t even realized that using something other than a DisplayPort adapter was even an option.

When this third round of display adapters arrived, I connected the Mac Mini to my home office ViewSonic monitor via the new miniDVI-DVI adapter, and bam! – instant video signal!  The Mac Mini booted up and worked just fine at that point. 

So I put sent the three remaining, unused display display adapters back to dvicetech using the envelope that he had provided.  I was seriously blown away by the customer service dvicetech provided – excellent, prompt, polite communication; smart troubleshooting suggestions; and multiple packages shipped back and forth at his own expense – which I daresay beat out even what I would have gotten from Apple directly, had I purchased a new Mac Mini.  At a minimum, I would have been spending $5+ in gas money per round trip to the nearest Apple store, which is about 30 minutes from my home.  My advice is when shopping for parts on eBay, buy from dvicetech if possible – I certainly intend to do so in the future!

I’m somewhat less happy with Apple, with all of the time and hassle it took me to get the new Mac Mini working.  When I use some more specific Google search terms like mac mini viewsonic no video, there are lots of other similar problem reports.  This wasn’t exactly an “it just works!” experience.  My takeaways are:

• For Mac video adapters, MiniDVI-DVI is a more reliable option than DisplayPort-DVI, although limited sample size is certainly a caveat here;
• Don’t assume everything will be instant sunshine and roses with Apple hardware, especially when interoperating between Apple hardware and standard-but-non-Apple hardware (although again, sample size caveats apply).

Still, in the end, the Mac Mini is now up and running, and I’m well along in coding up my new game in Xcode, Apple’s IDE for iOS.  It’s turning out to be a big project, so I’m several months away from even being ready to beta the app (particularly since I’m almost exclusively working in 90-minute-per-day chunks at the end of the evening after my kids are all in bed and other chores are all done), but I’ll certainly make an announcement here when I have something ready to show!

Workaround: ThinkPad docking station USB ports fail after dock

My current work PC is a Lenovo ThinkPad T530 running Windows 7.  I’m using it with a Lenovo ThinkPad docking station, the “Thinkpad Mini Dock Plus Series 3 with USB 3.0”. 

Occasionally, when I re-dock the PC, some or all of the USB 2.0 ports in the docking station do not “turn on” and work properly.   This most obviously manifests in my USB keyboard and/or mouse plugged into the docking station not working.  The keyboard’s indicator lights (Num Lock / Caps Lock / Scroll Lock) won’t turn on, and the red laser light on the underside of the mouse doesn’t come on.

Removing and re-inserting the keyboard and mouse USB cables into the docking station does not fix the issue.

The workaround that I found that does work (under Windows 7) is as follows:

1. Open Device Manager.  (From the Start menu’s Run field, type “device manager” and press Enter.  Obviously, use the laptop’s onboard keyboard and mouse if the external keyboard and mouse aren’t working.)
2. Scroll down to the bottom of the Device Manager window, and click on “Universal Serial Bus Controllers” to expand it.
3. Right-click on the first entry in the expanded list named “USB Root Hub,” and select Disable from the context menu.
4. Right-click once again on that same “USB Root Hub” entry, and select Enable from the context menu.

If it works, the list of items under “Universal Serial Bus controllers” should refresh, and you should see some additional items in the list.  The USB ports on the docking station and devices connected to them should be working once again.

I’ve never had the single USB 3.0 port on the docking station fail to work in this way, so another workaround might be to attach an external USB hub to that port, and plug all external USB devices into the hub.

I did try replacing the docking station with another copy of the same model, but the intermittent issue persisted, so the issue appears to be with the PC software or hardware itself, not with the docking station alone.

This workaround is inconvenient, but it beats rebooting!  Let me know if you know of a better workaround or a fix.

Mac Impressions - The First 60 Minutes

What follows is something a little bit different: For your amusement, a somewhat steam-of-conciousness recounting of my first 60 minutes with a Mac Mini (purchased for the purpose of doing some on-the-side iOS development), from my background as a guy with 25+ years of DOS/Windows experience (dating back to doing BASIC development on an IBM PC my Dad brought home from work when I was 6 years old!) and some limited Unix/Linux experience, but no real Mac experience at all. Enjoy!

8:01 - Successfully booted up! Logitech mouse and "brand X" USB Windows keyboard (in use temporarily until next reboot, when I'll try my PS/2 IBM Model M keyboard with an adapter) appear to be working!

8:03 - Using Safari... the mouse scroll direction is backwards! I can't cope with that, Googling for a fix... okay, fixed it in System Preferences | Mouse.

8:06 - Some keyboard shortcuts use Win (Command)! Some still use Ctrl (Ctrl+Tab to switch browser tabs!) Confusing! A lot to learn!

8:11 - Keyboard document navigation is different!
- “Move one word at a time” now uses Alt (Option), not Ctrl!
- What’s the shortcut to jump to the beginning or end of a document?

8:14 - Middle-click doesn’t close tabs in Safari!
- Maybe I’ll switch to Firefox later. But going to stick out the “native experience” for now.
- Also, where are the favicons on the tabs? Hard to tell them apart.

8:17 - Ctrl+Tab doesn’t use MRU order in Safari!
- To be fair, no other browser gets this right either by default except Opera. Will be a deal-breaker though if it turns out that there’s no workaround, just like Chrome.

8:23 - Woah - Ctrl+LeftArrow takes me to some other desktop that shows a calendar and a calculator and some other stuff! (Hit that by accident trying to navigate within some text in an editor!)

8:27 - Looking through system preferences, Ctrl+UpArrow = Mission Control apparently! Seems like an interesting Alt+Tab (er, Cmd+Tab) alternative?

8:30 - Mouse button 4 (thumb button) doesn’t go Back in Safari! Is there a way to make that work?

8:32 - Some things are somewhat familiar from iPod Touch, e.g. Notifications Center and FaceTime.

8:35 - Safari seems bad at suppressing popup windows? Or maybe I’m just wandering into more “grey” parts of the Internet as I’m trying to look up things like keyboard shortcuts and haven’t learned yet which sites have good Mac info? But I've seen at least 2 spammy-looking popup windows show up in front of me so far.

8:37 - Thinking about security as I'm having to type in a couple of my passwords... How do I know that this Mac isn’t running a keylogger? What’s the equivalent of Ctrl+Shift+Esc on this thing? (Just tried that; it didn’t do anything, nor did Cmd+Shift+Esc, not that I really expected it to. Maybe a good old command-line ps -ef is the answer, like in Linux... will need to learn what's "normal" on this system and what's not.)

8:38 - Ok, so how do I open a Terminal window? Searching Google... Aha, Cmd+Space is kind of an analog to Win+R (or just the Win key as of Windows 7) on Windows. Then I can just type “terminal” and hit Enter.

8:41 - Where’s the filesystem explorer? I remember that it’s called “Finder” on Mac. Poking around... Ah, I see, it’s activated from the smiley mac face at the far left of the “applications bar” thingy at the bottom of the screen.

8:42 - The Close button is in the top-left of each window, not top-right. Is there an Alt+F4 (universal “close window” shortcut key) on this?

8:45 - Talking to my wife on her iPhone via “Messages!” Man, this thing works flawlessly, unlike any solution that I’ve tried on a PC to chat with her on her iPhone. User experience is just like my iPod Touch. Impressed, but at the same time disappointed at that lack of a good PC solution for me to use at work; real keys (PC) blow away virtual ones (iOS device) every time.

8:48 - The Home and End keys don’t work in the Messages message entry field, but they do work in other places like Google Docs running in Safari -- what’s that about? They don’t seem to work in the Safari address bar either?

8:50 - Thinking ahead to writing up all this as a blog post. Is there an equivalent to LiveWriter on Mac? I'll probably just use the Blogger web UI for starters.

8:52 - Oh, found by accident that Cmd+UpArrow/DownArrow is like Ctrl+Home/Ctrl+End. Was trying to do that Ctrl+UpArrow thing to activate Mission Control to find out where my terminal window went. Don’t see it... I must have closed it. But now I also see that Cmd+LeftArrow/RightArrow act like Home and End.

8:56 - What’s Launchpad do... ah, seems reminiscent of the iOS desktop. *All* of the icons!

8:58 - No Menu key? Is there a “right-click” key on Mac to bring up spelling fixes when the caret is on a misspelled word, without reaching for the mouse?

9:00 - Typing in a Safari textarea field and typoed in “cusotmer” and Safari is showing a little popup with the suggested corrected spelling “customer” -- great! But how do I accept that correction without reaching for the mouse and clicking on it? Just pressing space bar like on iOS doesn’t seem to do the trick? I can’t seem to use the tab key to get to the little popup either? ... Ah, I need to hit DownArrow and then enter, ok.

...And there you have it, the highlights of my first 60 minutes as a Mac user! Maybe I'll come back to this topic and do a "Mac Impressions - The First 60 Days" and give some more in-depth impressions once I've gotten to spend some significant time with this new machine.

Finally, here's one bonus thought to leave you with:

10:40 - I must have hit "Home" and/or "End" at least 15-20 times as I was editing this post, and in the process discovered that those keys on a Mac apparently serve to scroll the viewport (but not the insertion point, thankfully) to the beginning or end of a long editable text area, respectively! The muscle memory from all those years of DOS and Windows is baked in pretty hard. It will be interesting to see how difficult it is to overcome that as I continue using this new Mac... at least *some* of the time already I'm remembering to use Cmd+Left/Right instead of Home/End, but it's far from 100% at this early stage!

ATG 10.1.2–“FileNotFound” error using Endeca search from an InvokeAssembler droplet

Issue

In Oracle ATG Commerce 10.1.2, a JSP page in my project failing to properly retrieve results from Endeca Guided Search.  (The application is not using Endeca Experience Manager).  The following error appeared in the ATG application log file:

**** Error      Mon Apr 15 09:55:55 EDT 2013    1366034155171   /atg/endeca/assembler/droplet/InvokeAssembler   A problem occurred assembling the content for content item /services/guidedsearch. The response received was {contentUri=/services/guidedsearch, @type=ContentInclude, @error=FileNotFound}. Servicing the error open parameter.

Resolution

The problem turned out to be not a missing file, but that some of the Endeca-related ATG components were misconfigured.  We resolved the problem by making sure that the following ATG component properties had correct values:

atg.endeca.ApplicationConfiguration

• baseApplicationName
• defaultApplicationName
• keyToApplicationName
• defaultLanguageForApplications
• workbenchHostName
• workbenchPort

atg.endeca.assembler.AssemblerApplicationConfiguration

• defaultMdexHostName
• defaultMdexPort

atg.endeca.assembler.cartridge.manager.DefaultWorkbenchContentSource

• serverPort

atg.endeca.assembler.cartridge.manager.AssemblerSettings

• experienceManager

Update August 7, 2013

My team today encountered this "FileNotFound" error again when setting up a new environment. Even after verifying the ATG configuration was 100% correct, the problem persisted. The Endeca JSP Reference Application ("orange application") also returned search results successfully, which demonstrated that the Endeca MDEX server cluster was working ok.

The fix turned out being that the promote_content.sh script located at endeca/apps/appname/control needed to be run. After running that script and having it report successful completion, ATG immediately starting being able to retrieve search results from Endeca successfully (no service restarts needed).