Activity for Chris Henson
-
Chris Henson posted a comment on discussion Help
I think you are looking for $(groupId) https://github.com/JumpMind/symmetric-ds/blob/f590c705d145519763dfe97146430c69368fc5e0/symmetric-core/src/main/java/org/jumpmind/symmetric/db/AbstractTriggerTemplate.java#L239-L247 Documentation mentions it here: http://www.symmetricds.org/doc/3.9/html/user-guide.html#_load_data Variables are discussed here: http://www.symmetricds.org/doc/3.9/html/user-guide.html#_variables We are always looking to improve the documentation. Please feel free to create pull requests...
-
We don't do much with Informix. Are you pointing out that when looking up tables on informix you need to use the informix.systables qualified by the owner? It looks like we just use the JDBC driver to look up metadata. JDBC has two concepts schema and catalog. They mean different things on different platforms. Have you tried to use catalog versus schema?
-
I'm not sure what you are asking. Can you maybe rephrase your question?
-
What database are you using? SymmetricDS has good bulk loaders for a few different databases. You also get some concurrency by creating multiple reload channels and putting them on different channel queues.
-
I would look at sym_data on node 2 to track down what happened. It should have a record of all the changes and where they came from.
-
The error_flag indicates that there was an error that occurred. It remains set until the status changes to OK. You see the LD status for multiple batches because they are sent according to max_batch_to_send. Acks don't come back until all batches in one sync are processed. Depending on your sync scenario you could set max_batch_to_send to 1 to get a more real time status update. You would want to set the push and pull jobs to run much more often if you do. If you monitor for the error_flag on either...
-
We tried this with your query and had no issues. Any other details that you can provide us would be great.
-
Did you set create.table.without.defaults=true at the source before the initial load was created?
-
The closest we have to an out of the box check is dbcompare. http://www.symmetricds.org/doc/3.9/html/user-guide.html#_dbcompare You need jdbc access to both databases in order to use the tool.
-
Sorry about that. It is checked in now. Adding the table at the clients would also be an option.
-
https://www.symmetricds.org/issues/view.php?id=3398
-
Ah. The clients being 3.5 makes sense why you see the error. A 3.8 client should not. To fix on the server side you need a patch. We'll fix this for the next point release.
-
During the load of a registration batch, the client is suppose to ignore events for tables that don't exist. Are you sure the failing clients are 3.8? If so, what version? There is a patch we can do to filter out sym_job on the server side. DataExtractorService.java line 238.
-
If you can get us enough information so that we can recreate the issue we'll fix it. Can we get your configuration, engine.properties settings, a dump of outgoing_batch where load_id={failed load id} and the count of the number of rows in the failed table and the symetricds log file?
-
If newer versions file.sync.enabled is defaulted to false. Is this your issue?
-
File sync is off by default. Have you enabled it using file.sync.enabled?
-
I have not heard about an issue like this before either. Your solution isn't clustered is it? Usually if the source node times out and sends the same batch again it will be rejected because the target node is already processing a request from the source. This was broken for a bit early on in 3.8 because of the introduction of queue on the channel. There were some timeout changes in 3.8 as well. On the server side, in sym_service.conf you can change: wrapper.app.parameter.2=--max-idle-time=90000 to...
-
3.8.31 is a bad release to be on in general because of this: https://www.symmetricds.org/issues/view.php?id=3318 Not sure if it is releated to your issue though. I thought issues like what you are describing had been addressed by this point with the parameter settings you have set. I would suggest trying 3.8.33 and if you still see the issue we'll help you get to the bottom of it.
-
https://sourceforge.net/p/symmetricds/discussion/739236/thread/ff67e20c/?limit=25#ed17
-
SymmetricDS should not create tables by default. You would have to do an initial load with initial.load.create.first turned on or insert into SYM_TABLE_RELOAD_REQUEST with CREATE_TABLE=1. Either way if create.table.without.pk.if.source.without.pk is set on the source node then primary keys should be left off. If they aren't then its a bug.
-
DB2 does support that property. You probably need to force the rebuild of triggers. symadmin -f sync-triggers
-
Can you post the log file of the startup of the CI node on 3.8.33? It sounds like maybe the engine didn't start ...
-
That parameter is only supported on some databases. What database are you using?
-
Do you have auto.reload=true? If so, I think outstanding batches get Ok'd when a store registers and the auto intiial load starts.
-
Try this out: https://github.com/JumpMind/symmetric-android-client-demo. More details as to what went wrong would be great. We do have users that sync Android.
-
We checked in changes to fix the process status. Not sure about your other issue of data not capturing. Can you check your sym_data_gap table to make sure the data_ids in sym_data are in the range found in sym_data_gap?
-
We took a note to test Android on 3.9. I don't think that has been done yet. Thanks for bringing this to our attention.
-
That is a bug that was introduced 3.8.24. I entered an issue for it. https://www.symmetricds.org/issues/view.php?id=3321
-
Try 3.8.32. There was a fix for corrupted common batches in there. The only reason you need the shared staging area is if you are doing initial loads and have extract in background turned on. I'd turn off extract in background if you don't have a shared file system.
-
You have both nodes set up as regsistration servers (registation.url is blank). You need to have one node register with the other. Choose one node for your registration node and on the other set the registration.url equal to the registration node's sync.url. On the node that you want to regsiter you'll have to wipe out sym_node_identity and restart the SymmetricDS instance. Because it willl no longer have an identity it and the registration.url is set it will attempt to regsiter using it's regst...
-
These forums are for open source users. If you need help with pro you should contact sales@jumpmind.com. From an open source perspecitive, I'd be interested in seeing your engines/.properties files and the log output of each node.
-
Is the same data being captured twice in sym_data at the source? A batch will not be loaded multiple times unless sym_incoming_batch was purged and the batch was resent for some reason.
-
Have you tried later versions of SymmetricDS?
-
Our foreign key ordering algorithm changed in 3.8.30. I think you are running in to a bug where a table that is configured for replication is referencing a table via a FK that is NOT configured for replication. If that is the case, you can work around the issue adding the referenced table to be sync'd. I'll enter an issue to be fixed in 3.8.31.
-
Connecting to an Azure version of Sql Server is pretty much the same as connecting to Sql Server. You might want to use the Microsoft JDBC Driver versus the JTDS JDBC Driver.
-
Did you setup jconnect so it works with the database? There are instructions here: http://www.symmetricds.org/doc/3.8/html/user-guide.html#_sybase_ase
-
What database? Early versions of 3.8 had some batch corruption issues. Maybe you should try to delete the batch from the staging area and let it reprocess.
-
For your master node, is the cluster.server.id unique for each replica instance? Each member of the cluster needs to have a unique cluster.server.id. If so, can you provide more details on your setup and how you are determining that data is "syncing multiple times"?
-
I think maybe the transitive dependencies aren't setup quite right. Try adding each of the symmetric artifacts like this: <dependency> <groupId>org.jumpmind.symmetric</groupId> <artifactId>symmetric-server</artifactId> </dependency> <dependency> <groupId>org.jumpmind.symmetric</groupId> <artifactId>symmetric-client</artifactId> </dependency> <dependency> <groupId>org.jumpmind.symmetric</groupId> <artifactId>symmetric-core</artifactId> </dependency> <dependency> <groupId>org.jumpmind.symmetric</groupId>...
-
I would be going after the root cause of the error to see if you can figure out how to adjust your configuration to avoid the errors. You can configure staging to timeout an extracted file after so long versus the default behavior of keeping the file around until it is purged. Set stream.to.file.purge.on.ttl.enabled=true and set stream.to.file.ttl.ms to the number of milliseconds you want the files to stick around.
-
get this stuff to compile on java 8
-
Yep. Probably should add a link to it on the github site: http://maven.jumpmind.com/repo/
-
You can turn off the management of tabels by SymmetricDS using this property by setting auto.config.database=false. This is on our list to address. We have several primary keys that should be smaller for databases with limitation on key size.
-
There is an extension point called IDatabaseWriterErrorHandler that can be implemented. It allows you to write code to handle the error. Here is a link on how to develop extension points: http://www.symmetricds.org/doc/3.8/html/user-guide.html#_extension_points Here is a link on how to register extension points in the database: http://www.symmetricds.org/doc/3.8/html/user-guide.html#_extensions
-
It is trying to insert into the sym_data_gap table. Your properties files look correct to me. cluster.lock.enabled=true should prevent two nodes from attempting to route at the same time. Could you server times be different causing one node in the cluster to break a cluster lock?
-
The url used to register is printed to the log file. You can grab that url and test from a browser to take the android client out of the equation and possibly the network.
-
I usually reccomend adding a node_id column to the central version of the database and setting the pk to be a composite pk. You can use a transform to add the node id.
-
Are each of the batches for different nodes? The batch number is in the file name. It might be the same captured data that was routed to multiple nodes.
-
Maybe the purge job is not running? Do you see errors in the log file?
-
As of 3.8 SymmetricDS keeps data in the staging directory (tmp folder) until the batch tables have been purged. This is the default behavior. You can switch it over to a purge the staging directory on a time basis by setting stream.to.file.purge.on.ttl.enabled=true. stream.to.file.ttl.ms controls how long files will be kept around.
-
Maybe you are being effected by this? https://www.symmetricds.org/issues/view.php?id=3174
-
Using JTDS and Sql Server requires the TCP/IP communications be enabled. Could that be your issue? https://technet.microsoft.com/en-us/library/hh231672(v=sql.110).aspx
-
Correct. Setting it to all would work.
-
There is a setting on sym_trigger called use_stream_lobs that might help you out. If you enable it, then lob data won't be captured and instead it is selected out of the database at synchronization time.
-
You would want to leave the ; out of the sql statement.
-
I think you need to uninstall the service and reinstall after you add the full path to java in the sym_service.conf
-
Some versions of the JDBC driver read the entire result set into memory. I am guessing if you upgrade the driver you'll get better results.
-
Did you stop, uninstall and start the engine in order to do the install? If you don't call start() on the engine again, then the tables will not be recreated after the install.
-
What version of SymmetricDS are you using? What is the state of sym_node and sym_node_security at both the server and the client before re-registration is attempted?
-
Can you give us an example transform and point out the unexpected case differences that you have had to deal with? Providing source and target table definitions would also help us understand the issue.
-
Are you doing the Quick Start or the Demo? If you are doing the Quick Start what database type are you using?
-
Was there any output in wrapper.log? In metl_service.conf maybe you need to give the full path to java for the wrapper.java.command setting ...
-
SymmetricDS is a Java based solution. It can only be "embedded" in other Java applications. That doesn't mean you can't run it side by side with OpenEMR. SymmetricDS does have it's own web server.
-
If you have a SymmetricDS pro license you can get support at support.jumpmind.com.
-
I'm not sure. I have never attempted to use SymmetricDS in an osgi environment.
-
@erilong is awesome, but not that awesome. I think he added support for Sql Server to Sql Server for now.
-
You can enable use_stream_lobs on sym_trigger so that SymmetricDS does not capture lob data and instead selects if from the original table when synchronizing.
-
The sql lite jdbc driver we use doesn't support the Arm devices. I suggest using the native c client.
-
Is there such a thing as a "remote" sqlite database? We do support Sqlite from both the native client and the java client.
-
If you have data in sym_data that is not batched, then it might be because it was inserted by a long running transaction. SymmetricDS will move past holes in sym_data after routing.stale.dataid.gap.time.ms (default of 20 minutes) time has passed. If a transaction takes longer than 20 minutes, then data could be skipped. If this happens you will see the following message: Found a gap in data_id at {}. Skipping it because the gap expired or Found a gap in data_id from {} to {}. Skipping it because...
-
The AndroidSymmetricEngine has an uninstall method on it
-
So you want to load 004 from 000? What is the sym_node_security row for 000 at 000 look like? The values for 004's sym_node_security from above are from the 000 database, correct?
-
SymmetricDS's create tables feature doesn't currently support cross schema foreign keys.
-
Are you using an osgi container? Maybe it has something to do with that.
-
Some database dialects all you to specify custom trigger text. It is also OK to insert directly into sym_data with a reload event.
-
I think I found the issue. I checked in a fix under this issue. After some more testing we will be releasing 3.8.23.
-
Are there other errors that happen prior to the corruption? Like maybe a network error or database error?
-
You should put your configuration in your engines/properties file versus tweaking symmetric.properties. We removed it because we didn't want people editting it.
-
If initial.load.create.first=true on the source system, then create events are queue'd up prior to an initial load. How did you kick off your initial load?
-
We typically use routers to subset data (versus trigger conditions). In routers you have access to both old and new column values to use for routing and transformations.
-
So the only issue is that a row is logged in sym_incoming_error? Was there an error in the log file that self corrected eventually?
-
We have an open issue to track this, but haven't had a chance to fix it yet. https://www.symmetricds.org/issues/view.php?id=3029
-
Maybe you need to set sendStringParametersAsUnicode=false in the jdbc url? We have seen poor peformance because the driver casts to nvarchar by default when binding variables to prepared statements.
-
Maybe you beed to set sendStringParametersAsUnicode=false in the jdbc url? We have seen poor peformance because the driver casts to nvarchar by default when binding variables to prepared statements.
-
What is the value of sym_node_security for the row where you queue'd up the load? If registration_enabled was left as 1 somehow or registration_time is null then loads won't be queue'd up. select registration_enabled, registration_time where node_id in (select node_id from sym_node_identity)
-
This does sound the same as 2984. Are you doing reloads or is this normal replication? What do you have stream.to.file.threshold.bytes set to?
-
Try setting initial.load.extract.and.send.when.staged=false. I think I just found a bug where this could happen with initial.load.extract.and.send.when.staged=true
-
I have not seen that happen before. Are you including a different version of Spring than SymmetricDS references?
-
When this happens is it always the first batch that is left in RQ status? Is there a log message about setting it back to RQ status? Does the batch actually exist in the staging directory?
-
Feel free to post an issue to the bug tracker (and possibly details on the workaround). We can work on a fix.
-
This is surprising. I thought had worked through most of the bugs with extract in background. Is the extract process stalling? Can you attach a full log file, the contents of sym_outgoing_batch for the load you are referring to and the contents of sym_extract_request?
-
This was answered here: https://sourceforge.net/p/symmetricds/discussion/739236/thread/55b72661/?limit=25#dc33
-
Are you using file sync? If not, then set file.sync.enable=false. If so, set file.sync.fast.scan=true. I think the initial load works more like you would expect when that is turned on.
-
It looks like the constructor of the LinkedBlockingQueue is passed the value of routing.peek.ahead.window.after.max.size. Could it be set to 0 or a negative value?
-
Both of those properties need to be enabled. In 3.8 they are enabled by default.
-
That is intentional. The encoding is platform specific. We use whatever type of encoding is easily availble in the database.
-
Is this just a one time migration? If you do an iniital load versus touching or inserting...
-
I entered an issue to improve the documentation. https://www.symmetricds.org/iss...
-
The columns are also bound in under uppercase variable names. You should be able...
-
Prem - This happens when the server is busy trying to load data. It could be that...
1 >