BaseLine Batch

BaseLine Batch can alert you to any security changes at any of the email addresses you send to. It is the third step in using CheckTLS to meet your email security needs:

  1. Check your own email system.
    Use the More Options features on the two tests on our home page, Check How You Send Email and Check How You Get Email to thoroughly test your email.
  2. Check that the email addresses you send to are secure.
    Use CheckTLS "Batches" to run //email/testTo: ("TestReceiver") on a number of email addresses and only report on a few details like ConfidenceFactor, TLS Version, etc.
  3. Know if anything changes.
    • For your own email system, monitor a Thru BatchTest targeting your own domain.
    • Monitor all the domains you send to with BaseLine Batch.
    "Screws fall out all the time, the world is an imperfect place."

That last step is important because you are responsible for protecting mail you send (See "The Responsible Party" section of Internet Email Security Made Simple)
and
your emailer will send plain text if the recipient asks you to (prove this with //email/testMandatoryFrom:).

BaseLine Batch

With BaseLine Batch, you run your Batch of email addresses once and tell CheckTLS to save the results. Then when you run it again, you only see the differences between today and the saved results (the "BaseLine").

BaseLine Batch makes checking email addresses 1000 times faster. Instead of periodically looking at every address, only look at the ones that have changed. A tuned BaseLine Batch typically flags one out of a thousand addresses.

At any time you can update a BaseLine score, so if a target fixes or improves their email, you can set the new score as their BaseLine.

With BaseLine Batch, you:

  • list all the email addresses you send to
  • test the addresses once and save the results (the "BaseLine")
  • focus/split the test
  • create the BaseLine test and run it once
  • review and tune the initial results
  • run and tune the BaseLine regularly

Using BaseLine Batch

BaseLine Batch uses a lot of XML files. Microsoft Notepad will edit XML files, or download Microsoft's XML Notepad (required Edge to download) or Microsoft's more advanced XML Copy Editor.

list all the email addresses you send to

Create a Batch of all the targets you want to BaseLine: <BatchTest TestType="setbaseline"> <Target>Dori.com</Target> <Target>Nori.com</Target> <Target>Ori.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest>

Be sure the TestType is "setbaseline".

We recommend:

  • Using Format=csv so you can load the results easily in Excel
  • Limiting a Batch to about 5000 Targets (use multiple batches)
  • Grouping addresses by type

test the addresses once and save the results (the "BaseLine")

Run the Batch and wait for it to finish.

focus/split the test

We recommend splitting the Batch into three separate BaseLine Batches: Production, Insecure, and Untestable. Focusing and splitting the test will save time in the future and improve your results. It is worth some manual effort.

Remove any addresses with typos or with results you don't want:

  • Download the csv results.
  • Open the csv in Excel and sort the results by Score:
    -1
    Invalid Targets that are not valid email domains so could never be sent to. Fix them (typos) or remove them.
    0-49
    Insecure Targets (did not do TLS)
    You should not be sending any confidential or protected information via email to these. Any email sent to them is going over the Internet in plain-text, which is likely illegal.
    Move them to a different Batch (Insecure) and test them on a different schedule. When/if one becomes secure and you start sending to it, you can move it to a Production Batch.
    50
    Unreachable Targets that we could not test. These could be typos or intermittent failures (see Vagaries below).
    Invalid addresses should be removed. Addresses that consistently are not testable should be moved to a differnet Batch (Untestable). Good addresses that are only infrequently unreachable should stay in the Production BaseLine.
  • Removed Targets will be automatically removed from the BaseLine when you next run it.
  • Infrequently unreachable (Score=50) Targets should be removed from the stored BaseLine Scores but not removed from the Batch. That way they will still test despite having intermittently been untestable. Use the RangeDelete option in //email/editBaseLines to remove all Targets with a score of 50.
  • Changed Targets should be updated in //email/editBaseLines. This where the matching scores are stored.
  • If you make a lot of changes to the Batch you should re-run the Batch to save a new BaseLine.

create the BaseLine test and run it once

Switch the Batch from setting the BaseLine to checking the BaseLine by changing the TestType to just "baseline": <BatchTest TestType="baseline"> <Target>Dori.com</Target> <Target>Nori.com</Target> <Target>Ori.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest>

Run the Batch and wait for it to finish.

review and tune the initial results

Running the BaseLine Batch compares the current state (Score) of a Target with the saved BaseLine state. The result is just the targets that have changed: "Target","BaseLineScore","CurrentScore","Match" "Nori.com","100","0","0" You can also use <Format>xml</Format>: <CheckTLS> <Results test="BatchTest_baseline" version="V03.17.09" format="xml" id="52" description="BaseLine compare" > <Result origin="www11-do.checktls.com"> <eMailAddress>Nori.com</eMailAddress> <BaseLineScore>100</BaseLineScore> <CurrentScore>0</CurrentScore> <Match>0</Match> </Result> </Results> </CheckTLS> Notice BaseLine test has a Match node so you do not have to parse and compare scores.

As with focusing the test above, taking some time now to tune the BaseLine will save time in the future and improve your results.

Once you have run a BaseLine, the following tuning can improve your results:

For really large Batches (thousands of Targets), see the next section "BaseLine Batch Auto-Tune".

run and tune the BaseLine regularly

When you want to compare the present settings with your saved BaseLine, run the Batch. As above, the result will be just the targets that have changed.

Periodically, if not every time, you should tune the BaseLine with new results. If a Target changes you should remove the old Match and add the new one, so future BaseLines will BaseLine against the Target's new Score.

If you ever want to completely reload all the BaseLines from this Batch, just reset the Format value and add the BaseLine attribute back and run the Batch. Just don't forget to put them back or you will replace the BaseLine every run instead of comparing to the BaseLine.

BaseLine Batch Auto-Tune

Auto-Tune automates the process of tuning large Batches. It removes mis-matches (failed BaseLine Targets) by adding new allowed matching scores for the mis-matches.

This is similar to re-running the Batch from scratch to reset all the BaseLines, but instead of replacing all the scores it adds new ones for any Targets that need them.

You should review the results of an automatic tune to make sure you don't inadvertently add a score that you do not want to accept. Targets effectively disappear from your exception report when you add their Score as a match.

To Auto-Tune a BaseLine Batch:

  • You must have run at least one BaseLine (i.e. a Batch with TestType set to "baseline" -- see above).
  • You must format the result as xml (i.e. <Format>xml</Format> -- see above).
  • Use the AutoTune button in //email/editBaseLines to open the AutoTune screen.
  • The Minimum Score depends on the purpose of this BaseLine (see Types of BaseLines below):
    Production Email
    90 and above (does good encryption)
    Insecure Email
    0 to 49 (does not do encryption)
    Untestable Email
    50 (unreachable, therefore untested)
  • Check the XML Online box. This will use the copy of your results that we have stored on our servers.
    See below for how to manually fine tune the Auto-Tune.
  • Click the RunTune button to perform the Auto-Tune.
  • Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons: added: weakdomain.com = 90 added: anotherweakdomain.com = 85; ... skipping duplicate: gooddomain.com = 90..100 added 274 records
  • Next time you run the BaseLine, the mis-matches you had will be gone.

Manual BaseLine Batch Auto-Tune

To Manually Auto-Tune a BaseLine Batch:

  • Download the last BaseLine Batch results (//email/editSavedTests:last).
    Not the original run, but an xml-baseline run, i.e. one that looks like this: <Result> <eMailAddress>weakdomain.com</eMailAddress> <BaseLineScore>60</BaseLineScore> <CurrentScore>90</CurrentScore> <Match>0</Match> </Result>
  • Load the results file into Excel, sort by CurrentScore, and note any lines you don't want to update.
  • Remove the records in an editor that understands XML (Excel will not let you save XML) or any text editor if you are careful to preserve the proper XML formatting.
  • Use the AutoTune button in //email/editBaseLines to upload your edited results file and then to run it.
  • Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons: added: weakdomain.com = 90 skipping duplicate: gooddomain.com = 90..100 added 1 record
  • Next time you run the BaseLine, the mis-matches you specified will be gone.

Types of BaseLines

We suggest you group email addresses ("Targets") that you want to test into three types:

  • Production
  • Insecure
  • Untestable

Remember that our Confidence Factor does not score the email address; rather it specifically scores the security of the email address.

Production Email

A BaseLine of Production Targets is a list of email addresses that you use that must be encrypted. You want to know if ever any one of them "breaks" and stops accepting encrypted emails. If one does break, you know to stop using it and contact them (maybe before they even realize they have a problem). If the Target is broken for a long time you may have to move it to one of the below BaseLines.

These should have Match Scores of 90 or above. Why accept a score that is measurably insecure? If you do find a Target below 90 that you want to accept, we allow you to add it.

Insecure Email

A BaseLine of Insecure Targets is a list of email addresses that you are not using because they are not encrypted. You want to know if ever any one of them gets "fixed" and starts accepting encrypted emails. Once they are OK you can let your organization know they can start using these email addresses. Then you should move fixed addresses to a Production BaseLine to make sure it stays secure.

These have Match Scores of 0 (zero) to 49. Note 50 is a special case below.

Untestable Email

A BaseLine of Untestable Targets is a list of email addresses that CheckTLS is unable to test. We don't know if they are secure or not, so we give them a "fifty fifty" chance. You want to know if ever any one of them gets "fixed" so CheckTLS can reach them and start testing them.

These have Match Scores of exactly 50.

Working Around Email Vagaries

Internet Email is designed to "never lose an email" and "get the mail through". It has enough redundancies to make sending any one email an adventure, as two emails sent at the same time can go two different ways. That makes it impossible to answer the question "Will every email ever sent to XYZ.com be properly encrypted?"

Which is why CheckTLS created the ConfidenceFactor. It takes everything into consideration to answer the question "How good is XYZ.com's email security?"

We call all the differences that an email system can exhibit "Vagaries".

Vagaries can result in a "false report" of a BaseLine change. The most common "false report" is when an email server gets busy and tells the sender to "try again later". This is expressly allowed by the formal email standard, and the sender is instructed to just try again in a few minutes. But our real-time test reports that as a failure to connect.

Other "false reports" come from one server in a redundant server pool having slightly different settings, or catching a site in the process of routine maintenance, etc. All these result in a slightly different Confidence Factor.

When doing BaseLine testing, we recommend TestType="receiverquick". "receiverquick" uses the Quick option in //email/testTo: to test only the one most likely MX host for a domain. This eliminiates most Vagaries.

Between any two BaseLine "receiverquick" runs we see less than 0.5% mis-matched Confidence Factors. So CheckTLS BaseLine testing makes checking your trading partners' emails 200 times easier.

A few simple tune-ups can reduce false reports to one in a thousand or less:

  • Use //email/baseline to set the score or scores that are acceptable.
    • Manually set the score
    • Add two or more scores
    • Add a range of scores
  • Move any domains with frequent "false positives" to a separate Batch, or remove them altogether. BaseLine doesn't work well for domains that change all the time.

Advanced Features

BaseLine testing has some additional features:

Multiple Scores and Score Minimums and Maximums

//email/baseline lets you list two or more scores that will match a Target. It also lets you set a scoring range that will match, so for example you can use the range 90 to 100 to mean "Yes this domain uses TLS".

Multiple BaseLines for a Single Domain

The standard //email/testTo: ("TestReceiver") test has many options. BaseLine Batch lets you check the same domain with different options. For example, you can BaseLine the domain's security with TLS version 1.2 and separately with TLS version 1.3: <Target SSLVersion="TLSv1_2">V1.2@XYZ.com</Target> <Target SSLVersion="TLSv1_3">V1.3@XYZ.com</Target> This will show you when XYZ.com changes from TLS v1.2 to TLS v1.3.

BaseLine Batch keeps the full Target email address so you can BaseLine the same domain with different settings.

BaseLine and Details in the Same Batch

With a simple hack you can use a BaseLine Batch to show details like SSLVersion and Cert expiration for the same Targets that are in the BaseLine. Keep two nodes at the bottom of the Batch, one that makes it a BaseLine and one that makes it show details. Comment out one or the other to control what the Batch does. For example: <BatchTest TestType="receiverquick"> <Target>checktls.com</Target> <Target>Dori.com</Target> <Target>Nori.com</Target> <Target>Ori.com</Target> <Delivery> <Format>xml-baseline</Format> </Delivery> <!-- <Delivery> <To>file</To> <Format>xml-certdetail</Format> <OnlyNode>eMailAddress</OnlyNode> <OnlyNode>ConfidenceQFactor</OnlyNode> <OnlyNode>//MX[1]/@exchange</OnlyNode> <OnlyNode>//MX[1]/SSL/SSLVersion</OnlyNode> <OnlyNode>//MX[1]/SSL/Cipher</OnlyNode> <OnlyNode>//MX[1]/SSL/Certs/Cert[@number=1]/NotValidAfter</OnlyNode> </Delivery> --> </BatchTest>

WebService (API)

You can maintain the stored BaseLines for a Batch by using BaseLineEdit as a WebService.

ADD A TARGET:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODE=me@mydomain.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &TARGET=three@checktls.com &SCORE=99 &ACTION=wsAddTarget
Success returns: <Results> <Result>Success: three@checktls.com(#52) = 99</Result> </Results>
Any errors returns: <Errors> <Error>Invalid CUSTOMERCODE or CUSTOMERPASS: See FAQ</Error> <Error>Invalid BATCHID: 1a</Error> <Error>Invalid TARGET: (blank)</Error> <Error>Invalid SCORE: alpha</Error> </Errors>
DELETE A TARGET:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODE=me@mydomain.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &TARGET=three@checktls.com &ACTION=wsDeleteTarget
Success returns: <Results> <Result>Success: 1 record deleted</Result> </Results>
DELETE ALL TARGETS:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODE=me@mydomain.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &ACTION=wsDeleteAllTargets
DELETE BY SCORE:
Delete all records with a ScoreMin between 0 and 10 and a ScoreMax between 20 and 30: https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODE=me@mydomain.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &SCOREMIN=0..10 &SCOREMAX=20..30 &ACTION=wsDeleteScores
To delete all records with a ScoreMin below 50 use: &SCOREMIN=0..50 &SCOREMAX=0..999
Success returns: <Results> <Result>Success: 437 records deleted</Result> </Results>