Currently we use 2 object servers for load balancing. This means that a user that logs in can be directed to either object server. On a number of occasions users see discrepancies in the data when they are logged into different object servers. eg. a gl account created by a user logged into AOS1 cannot be seen by users logged into AOS2. Both AOS point to 1 DB Server, and both AOS are pointing to the application files located on AOS1. This has happened a number of times and there is no pattern to its occurance. We’ve been advised to check the AOT caching properties of the table affected to ensure that it is set to None. However, this problem has also affected tables where the Caching has already been set to None. The only way we’ve been able to get around this problem is by stopping and restarting the AOS. We’ve also been advised to delete an index file in the standard folder of the application files. This helped with the first instance of this issue. What we would like to know is if anyone has been involved in organisations similar to ours where there are multiple AOS, who have experienced this issue, and have been able to resolve it. We would appreciate your help. Thanks, Leana
Hi Which version of Axapta are you running? Before 3.0SP2, there were issues with load balancing. Are you running Windows Load Balancing or AOS clustering?
We are running version 3.0 SP3 and have the AOS Clustering setup.
hi what do you mean by: "Are you running Windows Load Balancing or AOS clustering? " I think that Load Balancing is a mechanism that is availabe through AOS clustering and you don’t have a choice what to use: load balancing or clustering??? Slawek
Hi Slawek ! Axapta AOS clustering is not same as Windows load balansing. AOS clustering is very simple: the AOS which has least users takes the next client. The actual workload does not matter. If some AOS hangs, then its clients are dropped and not moved to other AOSes. br,
Hello! We have had another incident of this issue just recently. I added a Budget Model (GL) and it was visible to me on AOS1, but User on AOS2 couldn’t see it. I ran the “Refresh Data” option under Tools > Development Tools > Application Objects > Refresh Data. This did not fix the problem. We reset the servers and this fixed the problem. But as you know, we cannot do this all the time. Nor do we know where else this may happen without us realising. It jeopardises the integrity of the data we pass onto our clients… Can you tell me if there are certain tables that need to have the Caching set to None? Or other places where they also have multiple AOS who have also had this issue? Thanks for your help, Leana
Hi One tip (this is no solution to the problem but something you can do): Duplicate all the “Refresh …” menu items (Refresh AOD, Dictionary, Data) and make them to be run on Server. Add those 3 new menu items to the Tools Menu. If you start them, the e.g. data will be refreshed on the AOS your client is running with.
I will try as you’ve suggested. Thanks very much for your help.
Hi, We are having the same kind of problems with two Object Servers running for one environment (company). When one Object Server clears the cache, does this normally would effect the cache on the other AOS? Thanks for all replies, JRA
Hi there, I am not sure whether if you clear the cache in one AOS, it would clear the cache in the other AOS as well. Mainly because I do not see any trigger that would clear the cache of second AOS. Can some one correct me if I am wrong here please. But I have a suggestion - if your installation do not have to run 24x7, you can schedule the AOS to restart some time (say early in the morning) thereby clearing the cache. Regards, Harish Mohanbabu
Thanks for your reply, Harish. I really need to be sure whether the cache clearing on an AOS is triggered by the cache clearing on other Object Servers. Does someone know a conclusive answer? The suggestion to schedule for the two AOS servers to restart once per 24 hrs. and thereby clearing the cache, is as far as I know not needed. I’ve read that all cache will be cleared automatically at least once in every 24 hrs. Regards, JRA
Hi, Yes - the official document states that all ‘Entire table’ caches are flushed every day around mid night. Unfortunately in my experience it doesn’t always work 100% [:(] Also I have noticed that the entire application cache gets refreshed only on restarting the AOS. Regards, Harish Mohanbabu
Hi From 3.0SP2 on, a new caching mechanism for Entire Table Caches was “re-introduced” (MBS first introduced it for 2.5SP4 and “forgot” it then for 3.0) The cache is synchronized every 60 seconds by default, but you can change that by setting the Max.cache sync. time parameter on the AOS settings.
Hi Helmut, You wrote: “The cache is synchronized every 60 seconds by default, but you can change that by setting the Max.cache sync. time parameter on the AOS settings.” Question: What if the field for the Max. cache sync time parameter is left blank? Perhaps this disables the synchronising mechanism??? This parameter is left/made blank on our AOS machines and both AOS’s don’t seem to get in synch at all. We do have version 3.0-SP2. Regards, JRA
Where do you change the <Max.cache sync. time parameter> also, how do you clear the cache of an AOS server manually? How do you clear the cache on your local machine manually? Thanks
How do you change the < Max Cache > parameter. How do you manually clear the cache from the AOS or the Client Thanks
How do you change the < Max Cache > parameter. How do you manually clear the cache from the AOS or the Client Thanks
How do you change the MAX CACHE paremeter How do you manually clear the cache on an AOS server or Client Thanks
You can clear the cache manually by stopping the AOS. The “Max Cache Synch Time” parameter can be found on the tab “database” in the settings window of an AOS in de Axapta Server Manager. Regards, JRA
I have tried to develop a model how Axapta scales. I am not sure if it is right, and I would appreciate some feedback. These are the sources I used: - Axapta Online Help (available on Microsofts website) - this forum (do a full text search for AOS) - The official Axapta sizing tool Axapta Architecture If you have more than, say, 20 Users, Microsoft recommends to set up Axapta in 3-tier thin client mode. That way, your clients will connect to an Axapta Object Server (AOS), which in turn will connect to the database. If one AOS instance is not enough, AOSs can be clustered. There is no defined upper limit for the number of AOS instances or the number of concurrent users. Published Facts about Axapta Scaling From the sources I listed, you can get these facts: - For your AOS, you need 1 GHz processor speed and 1 GByte RAM per 100 Users. - One single AOS instance can support no more than 182 Users. - Microsoft recommends a network architecture with database server – switch – AOS Cluster – switch – Axapta Clients. In other words, the AOS cluster has its own subnet. - The AOS instances in a cluster will coordinate themselves on UDP port 2712, using the proprietary AOCP (Axapta Object Communication Protocol). - An Axapta client will normally cache 50 records per database table. An AOS caches 1000 records per table. - Axapta has problems to synchronize the AOS caches in a Cluster. Microsoft introduced a workaround with SP2: each AOS will refresh its cache completely at midnight. - An AOS does not notice when a user kills his Axapta client using the Windows Task Manager. A user would do this if his client hangs. If this is done often, no new users can log on, because the user licences on the servers are used up. This is one reason why many Axapta system administrators restart their AOS cluster every night. - An Axapta installation scales mostly via the database server. An installation with more than one AOS will need a very fast database server. - If you have more than 400 concurrent users, the Microsoft Axapta sizing tool will not work. A MBS Partner is compelled to let Microsoft do the sizing. How many AOS will I need? Looking at the published facts, we arrive at some simple conclusions. - You should have the smallest number of AOS instances possible. If not, you will have more problems with cache inconsistencies than necessary. - One AOS instance will support no more than 182 concurrent users. It will then require 2 GByte RAM and 2 GHz processor speed. - You should dimension your AOS cluster so that it supports all licensed users at once. You should have more user licences than your maximum of concurrent users. - You should restart your AOS cluster every night (using net stop and net start). Example: You have 2000 licensed users. Your AOS machines have 2 GHz processors and 2 GByte RAM. You will need 2000 / 182 = 12 AOS instances, each on its own machine. A model for computing AOS performance Using the published facts, we can make up a model of how Axapta scales. Warning: This is my own private model, and I did NOT verify it thoroughly. I think it complies with the known facts, and delivers a good explanation for Axaptas behaviour. But it may be wrong. First, let us make some deductions: - The AOSs in a cluster have to synchronize their caches. If they would not, the users logged on to different AOS would experience data inconsistencies. The AOSs will synchronize using UDP 2712. So probably an AOS will send a broadcast when a record has changed, telling everybody that the record with the recid xxx has changed. This would explain why the AOS cluster should have its own subnet. This would explain also why caches fail to synchronize and why Microsoft does not succeed to fix this problem: UDP does not guarantee that the data arrives at the target. - Second, we can assume that an AOS will refresh its cache, each time it finds out a cached record has changed. A typical Axapta form will let every user see, for instance, the same 10 newest invoice headers. So if all users are busy entering invoices, all AOSs will need the same 10 invoices. So we arrive at this computation model: (1) Number of users: 360 (2) Updates per user and hour: 50 (3) Number of AOS instances: (1) / 182 + 1 = 2 (4) Number of users per AOS: (1) / (3) = 180 (5) Updates per AOS and hour: (4) * (2) = 180 * 50 = 9000 (6) Cache updates per AOS and hour: [(3) – 1] * (5) = [2 – 1] * 9000 = 9000 (7) Cache updates per Database and hour: (6) * (3) = 9000 * 2 = 18000 Now let’s do the numbers for 2000 users: (1) Number of users: 2000 (2) Updates per user and hour: 50 (3) Number of AOS instances: (1) / 182 + 1 = 12 (4) Number of users per AOS: (1) / (3) = 167 (5) Updates per AOS and hour: (4) * (2) = 180 * 50 = 8341 (6) Cache updates per AOS and hour: [(3) – 1] * (5) = [12 – 1] * 8341 = 91659 (7) Cache updates per Database and hour: (6) * (3) = 91659 * 12 = 1098901 So if you have 360 users, you have 18000 additional database reads per hour which are only needed for keeping cache consistency. These database reads have nothing to do with processing your business data. If we have 2000 Users, we arrive at an enormous 1.1 million extra read operations per hour, or 300 per second. This makes clear why Microsoft says an Axapta installation scales via the database server. So what can you do about it? - If you have lots of small Axapta installations, each with its own database, then you don’t have a problem. You can replicate the changes nightly, if you stop your AOS beforehand. - If you have more than 400 users on a single database, you should change your most frequently used forms, so that every user sees only his private data. Use Journals wherever possible. - Web programming is no solution. If you do your web programming with Axapta, it will use AOS none the less. If you don’t, and use the database directly (this is possible: you just have to increment the recids in the syssequences table), then you will circumvent the AOS caches and produce lots of cache inconsistencies.