AntiSamy 1.1 is out!

March 22, 2008 arshan dabirsiaghi 42 comments

I’m happy to say that the OWASP AntiSamy 1.1 Java API is officially out! Thanks to everyone on the OWASP AntiSamy mailing list for helping me get a better API out the door. There were really only 5-6 changes worth getting excited about. Here are the highlights:

Removed accidentally included internal Sun JRE classes (com.sun.*) and replaced with Xerces classes. this fixes the NoClassDefFound error you’d encounter on non-sun JREs (like in IBM WebSphere)
Re-factored code to remove reliance on setUserData() method to allow the code to run in Java 1.4
Escaped ‘#’ in onsiteURI regular expressions to address a known bug in the JVM which interpreted the hash mark as a comment character inside of character class definitions, e.g. [0-9A-Z,/#]. This flaw allowed disallowed protocol URLs (credits to Richard Rodger of Ricebridge Software for discovering this)
Changed code comment accidentally crediting HTMLCleaner with the cleaning – should be NekoHTML!
Also, we’ve got the JavaDocs online at Google!

The API is in a good place, and it’s getting the attention of a lot of people. I said from the beginning that if 10 people found AntiSamy useful I’d consider it a success. Considering the thousands of times AntiSamy materials have been downloaded, I really couldn’t be happier about how things have turned out.

However, I don’t want stay in just the Java world. Early goals for AntiSamy included getting a .NET and PHP version ready by Spring 2008 – that’s still very possible, but not without help. Luckily, it appears that the Stas Malyshev from the Zend group (think OWASP Incorporated, mega-focused on PHP) is looking to start the project. I planned to do the .NET version but I’ve been mega busy with other research. Hopefully I can find some OWASP Summer of Code money to get this done! Anybody from the OWASP .NET project feel like collaborating?

security antisamy, input validation, owasp, rich input validation, samy, security, webappsec, xss

#1 | Written by Serge about 5 years ago.

Hello Arshan,
First of all thank you for unicode updates.

I just found that all example policies allow:
a href=”javascript:alert(‘hello_samy’);

but filter: a href=”javascript:alert(‘hello samy’);

Also I would like lo ask if there a policy example that allows to load only onsite content for images, backgrounds and so on.

Thank you,
Serge
#2 | Written by arshan dabirsiaghi about 5 years ago.

Serge,

Thanks for your interest in the project! I can’t duplicate your tests, unfortunately. Can you give me some more details about your test environment?

Did these tests work in your copy of 1.0 as well? I don’t see your tests working on my test page (which is still 1.0, admittedly):

http://i8jesus.com:9080/AntiSamyDemoWebApp/test.jsp?profile=%3Ca+href%3D%22javascript%3Aalert%28%27hello+samy%27%29%22%3Etest%3C%2Fa%3E&policy=antisamy.xml

http://i8jesus.com:9080/AntiSamyDemoWebApp/test.jsp?profile=%3Ca+href%3D%22javascript%3Aalert%28%27hello_samy%27%29%22%3Etest%3C%2Fa%3E&policy=antisamy.xml

Maybe your JVM has some regular expression weirdness going on. Let’s figure this out!
#3 | Written by arshan dabirsiaghi about 5 years ago.

Oh yeah, I forgot to even mention the fact that AntiSamy now properly handles all those strange unicode characters.
#4 | Written by Serge about 5 years ago.

Arshan,
I use Railo (railo.ch) This is a CFML engine. I run it under Resin 3.1.2 on Mac OS 10.5.2

My Code looks like:

Click on this!

I will also check that on my production server and report here.

JVM info is:

java -version
java version “1.5.0_13″
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05-237)
Java HotSpot(TM) Client VM (build 1.5.0_13-119, mixed mode, sharing)

Thank you,
Serge
#5 | Written by Serge about 5 years ago.

UPS… some form problems here
Here are tags without the brackets:

cfset sanitizer = createObject(“java”, “org.owasp.validator.html.AntiSamy”)
cfsavecontent variable=”dd”

a href=”javascript:alert(‘hello_samy’);” Click on this! /a

cfset myResult = sanitizer.scan(dd,”antisamy-slashdot-1.1.xml”).getCleanHTML()

/cfsavecontent
#6 | Written by arshan dabirsiaghi about 5 years ago.

All the Mac people have left the office already, lazy jerks. I can try this out on Wednesday, but I can already tell you it’s probably the fact that your JVM handles regular expressions differently from the more popular JVMs like IBM or Sun.
#7 | Written by arshan dabirsiaghi about 5 years ago.

Another thing I might suggest (although you’ve probably done this) is getting a fresh policy file. Could it have been somehow altered by accident?
#8 | Written by Serge about 5 years ago.

No, I use antisamy-slashdot-1.1.xml
I’ll check that on server tomorrow and report here.

Thank you,
Serge
#9 | Written by Serge about 5 years ago.

Arshan, I affraid it is not JVM related issue.

Please check this:
http://supremedesign.ru/antisamy/

This either CFML or Railo specific problem.

What is the pattern that is responsible for that?

Thank you,
Serge
#10 | Written by arshan dabirsiaghi about 5 years ago.

Can you post the exact code page for that test page?

I put in: “[script]test[/script]“, and my result was:
“[invalidTag]test[/script]“. This is definitely not normal AntiSamy behavior.
#11 | Written by Serge about 5 years ago.

Arshan, here is the code: http://supremedesign.ru/antisamy/index.html
just hit view source or save the link as.

default.xml policy file is renamed antisamy-slashdot-1.1.xml and it’s located in the same directory as .cfm file.

If you wish you could get Railo at http://www.railo-technologies.com/en/index.cfm?treeID=224

I use Railo Server with JRE, but I guess Railo Express should behave in the same way, just place antisamy-bin.1.1.jar to railo-home/lib

Thank you,
Serge
#12 | Written by Serge about 5 years ago.

I guess script > invalidTag was Railo script protect behavior.
By default Railo “script-protect checks in all scopes for external data (cgi,cookie,form,url)”
#13 | Written by Serge about 5 years ago.

Arshan, may we conclude that:

1.
The problem appears in Railo and it doesn’t depend neither on JRE/JDK version we use nor on AntiSamy version we use.

2.
Pure AntiSamy jsp calls work Ok not regarding the version. But in Latin-1 environment.

So (as far as I’m not java guy, I can’t be 100% sure) this is most luckily Railo compiler/runtime problem?

Thank you,
Serge
#14 | Written by Serge about 5 years ago.

I replaced onsiteURL expression with .+(:)
For the moment it seems to be able to handle all http://ha.ckers.org/xss.html examples that I have tested.

I know it’s far from being ideal but for now it’s ok.

Also I have found that both url filters aren’t able to process non-latin characters.

Is there an option to enable them? For example I may need it for url’s like site.com?search=поиск

Thank you,
Serge
#15 | Written by arshan dabirsiaghi about 5 years ago.

I’m glad you found a temporary solution, and it does appear likely that this is a Railo problem, but I, unfortunately, have no easy way to confirm that.

As far as latin characters inside the URL, that is indeed tricky. I will need to figure out what the right solution for that is.

My knowledge of character sets is not that great. Right now the regular expression is something like this: [a-zA-Z]. Do you know a way to include non-latin characters while still excluding all other ASCII characters?
#16 | Written by Serge about 5 years ago.

Arshan,
Since I focus on client-side and UI I can’t be 100% sure, I guess this is about unicode.

So, relying on intuition, maybe ASCII to Unicode conversion will do the thing, or may be there is a way to permit unicode blocks:

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]/* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */

(I spotted that with Google)
#17 | Written by Serge about 5 years ago.

Arshan,
I did some experiments.

1. Railo works Ok with “([\w\\/\.\?=&;\#-~]+|\#(\w)+)” onsiteURL expression if we just pop “~” out from it. Unicode problem however exists.

2. This is unicode expression that seems to be work for onsiteURL:
([\p{L}\p{N}[\p{Zs}\t][^\P{C}]&/\#-.?=_~])*

(It’s not final I’m still playing with it)

I just lack hacking knowledge to test if it protects well. script:action constructions don’t pass the expression.

Here are some unicode links I have used:
http://www.regular-expressions.info/posixbrackets.html
http://www.regular-expressions.info/unicode.html

Here is the webform links:
http://supremedesign.ru/antisamy/
http://supremedesign.ru/antisamy/index.src
http://supremedesign.ru/antisamy/default.xml

Thank you for being so careful about the case,
Serge
#18 | Written by Serge about 5 years ago.

Little update

([\p{L}\p{N}\p{Zs}/\.\?=&;\-~])+

This expression seems to work very close to original one but supports unicode.

just # mark causes problems. I’m looking if there is a way to type as utf-8 code.

for example \p{#-code}

Thank you,
Serge
#19 | Written by arshan dabirsiaghi about 5 years ago.

Serge,

I am glad to see that Java allows the \p{X} marker. I will have to integrate this into a patch for the 1.1.1 policy files. I believe you can also specify the hash marker unicode value directly, we just have to figure out what it is.

It will be some work to update ALL the regular expressions, so I urge you to have some patience as I think through them. Preventing XSS is the first concern, although usability is a close second.

Thanks for making AntiSamy a better product! I’ll keep you posted on how I get through this.
#20 | Written by Serge about 5 years ago.

Hello Arshan,
I found that [img src="virus.js"/] is able to pass validation.

Is it dengerous?
#21 | Written by arshan dabirsiaghi about 5 years ago.

No, that’s not dangerous. Browsers won’t execute content they get for an image. The only dangers from tags are src and event handlers.
#22 | Written by Serge about 5 years ago.

yes, Arshan, thank you. I guess I was too tired to think
Actually it’s up to browser. One can always save .js as a .gif or inject some stuff into image directly… Html parser can’t handle this.

Here some good news: I’m testing Antisamy with slightly modified myspace policy file. It wasn’t much work to rewrite url and title patters for unicode. Other stuff is just fine filtering ancii.

You can check it at: http://blog.supremedesign.ru/xss

It’s nice to see how it handles thing like

[[/]script>alert(“;-(“);[[/]/script]
[p]:-)[/p]
#23 | Written by arshan dabirsiaghi about 5 years ago.

Serge, I’m glad it’s working out for you. Can you send me your policy file? Did you simply replace all [a-zA-Z] with \p{L} ?
#24 | Written by Serge about 5 years ago.

Hello Arshan, actually I plan to use 2 policy files.

Default policy, for comments and advanced for topics. I took myspace policy and modify it as I go. You can get it here: blog.supremedesign.ru/tmp/advanced.xml

Yes, I replaced [a-zA-Z] with \p{L} for the attributes that I expect to be filled with international content (src, title, alt). And disabled tags that I don’t expect to allow (form tags, style).

Also I would like to thank you once again for such a smart product!

Do I get it right that Antisamy recreates input as a DOM object and applies security policy to it? (I would like to blog about Antisamy)

Thank you,
Serge
#25 | Written by arshan dabirsiaghi about 5 years ago.

Yeah, this is the general process:
1. I use nekoHTML to translate the input into a predictable format
2. Validate the XML DOM (the output of step #1)
3. Serialize the XHTML/HTML using Apache libraries

Check out the slides:
http://owaspantisamy.googlecode.com/files/OWASP%20Fall%202007%20San%20Jose%20-%20Anti%20Samy.pptx

There’s also a paper for the gory details:
http://owaspantisamy.googlecode.com/files/Arshan%20Dabirsiaghi%20-%20Towards%20Malicious%20Code%20Detection%20and%20Removal.PDF
#26 | Written by Serge about 5 years ago.

Arshan, I just found that pdf example causes paser exeption:

This is some
text.

at org.cyberneko.html.parsers.DOMFragmentParser.startElement(DOMFragmentParser.java:431):431
at org.cyberneko.html.filters.DefaultFilter.startElement(DefaultFilter.java:179):179
at org.cyberneko.html.filters.NamespaceBinder.startElement(NamespaceBinder.java:286):286
at org.cyberneko.html.HTMLTagBalancer.callStartElement(HTMLTagBalancer.java:1009):1009
at org.cyberneko.html.HTMLTagBalancer.startElement(HTMLTagBalancer.java:639):639
at org.cyberneko.html.HTMLScanner$ContentScanner.scanStartElement(HTMLScanner.java:2407):2407
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:1881):1881
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:809):809
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:478):478
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:431):431
at org.cyberneko.html.parsers.DOMFragmentParser.parse(DOMFragmentParser.java:164):164

This happens on both my and your this happens on all test websites including yours. I guess this because of the linebreaks.

Thank you,
Serge
#27 | Written by Serge about 5 years ago.

Oh, sorry this is the code:
This is some
text.
#28 | Written by Serge about 5 years ago.

UPS.

[meta name=”expires;-
1;www.phishingsite.com”]This is some
text.[/meta]
#29 | Written by Serge about 5 years ago.

I’ve fixed that for a while with: reReplace(arguments.input, “[[:cntrl:]]”, chr(127), “all” )
#30 | Written by Serge about 5 years ago.

gg
#31 | Written by Serge about 5 years ago.

Sorry, again — [div- -lolz="fff\" ]gg[/div] causes parsing exeption too
#32 | Written by Marc about 5 years ago.

Hello all,

the problem mentioned above still seems to exist in 1.1.1. I was trying the slashdot 1.1.1 xml policy but still hrefs like “javascript:alert(‘xss’)” were accepted. Since I cannot see a “:” in the original onsiteURL, I’m not sure about the reason. However, by chaning the regex for onsiteURL from
([\p{L}\p{N}\\/\.\?=&;\#-~]+|\#(\w)+)
to
([\p{L}\p{N}\\/\.\?=&;\#-~&&[^:]]+|\#(\w)+)
all onsiteURLs containing a : are disallowed now (cf. character class subtraction in JavaDoc of java.regex.Pattern).

I hope I did not miss anything.

Regards
Marc
#33 | Written by arshan dabirsiaghi about 5 years ago.

Marc,

That is real strange. Can you put you enter an issue on Google?

Thanks,
Arshan

http://code.google.com/p/owaspantisamy/issues/entry.
#34 | Written by Marc about 5 years ago.

Hi Arshan,

the site seems to be inaccessible for me. It keeps saying “Your client does not have permission to get URL /p/owaspantisamy/issues/entry from this server”, although I created a Google account (and logged in, of course . Any suggestions? I could pass you a Java test class if you wish.

Regards
Marc
#35 | Written by Marc about 5 years ago.

Erm, now Google is working all of a sudden, sorry. I’ll enter an issue.
#36 | Written by Marc about 5 years ago.

And by the way, not all HTML tags that are advertised in the comments of antisamy-slashdot-1.1.1.xml are really supported. Missing definitions: strong, em, dl, dt, dd. No big deal, since I need to adjust the file to my specific requirements anyway.

Cheers
Marc
#37 | Written by Amit about 5 years ago.

Is it possible not to allow any html tag in user input. I tried using all the provided xml’s with AntiSamy code, but none of them is stopping for entering non-malicious html tags.

I am unable to find what to modify in the XML files so that i starts filtering out all the html tags.

Can anyone help?

Thanks
#38 | Written by arshan dabirsiaghi about 5 years ago.

You mean you’re trying to use AntiSamy to filter out ALL HTML? You don’t need AntiSamy unless you want to allow some rich input. Otherwise, all you have to do to be safe is HTML encode your input.

http://www.owasp.org/index.php/How_to_perform_HTML_entity_encoding_in_Java
#39 | Written by hitesh karel about 4 years ago.

Hi Darshil,
I have used Antisamy.I am facing one issue.When i am using this project with Tomcat server it is working fine and when same project is deploying at weblogic it throughing following execption at very start of application

Please find below exception and reply me that ASAP

[ServletContext(id=31644075,name=br248,context-
path=/br248)] Root cause of ServletException,
which the Web application container caught while
servicing the request.

Cause
The Web application container caught an unexpected
exception.

Action
Check the exception for the exact error message.

java.lang.Throwable: setPreserveEmptyAttributes
at org.owasp.validator.html.scan.AntiSamyDOMScanner.scan(Unknown Source)
at org.owasp.validator.html.AntiSamy.scan(Unknown Source)
at com.checkfree.wps.util.EscapeSpecialChars.replaceRestrictedHtmlTags(EscapeSpecialChars.java:79)
at com.checkfree.wps.util.CFHttpServlet.replaceRestrictedHtmlTags(CFHttpServlet.java:2225)
at
#40 | Written by arshan dabirsiaghi about 4 years ago.

To close the loop on this, I sent this e-mail to Hitesh and it fixed his problem:

>This is a classpath error. Somehow, WebLogic is finding an
>incorrect version of Xerces. First, check the MD5 or file
>size on the jars in both Tomcat and WebLogic and make sure
>they are the same. If they are confirmed to be the same, then
>we know that WebLogic is clobbering your Xerces with its own.
>This is a known issue in WebLogic. Check the following link
>and hopefully the instructions there will resolve your issue.

>http://edocs.bea.com/wls/docs81/faq/xml.html
#41 | Written by Abhishek about 3 years ago.

I have been trying out anti-samy. I know we can allow some tags. But can we make the default action encode. Because i am making an email page. i want to allow some tags for formatting and encode the rest to display the content without removing anything.
#42 | Written by arshan dabirsiaghi about 3 years ago.

There’s no way to change the default action, sorry.

omg.wtf.bbq.

AntiSamy 1.1 is out!

Leave a Comment

Recent Posts

Browse by Tags