- these work in conjunction with ACLs, however they are executed BEFORE the ACLs
- data filtration is a 'deny' principle whereas ACL a 'grant' principle
- data filtration reduces the need for scripting
- they run AFTER before query business rules
- will still see the 'removed due to security contraints' message unfortunately
- requires security_admin role just like ACLs but there is no admin override feature
- specific to scoped app it's defined in (might not have access to certain global/other app tables if defined in a scoped app)
- key design criteria: machine enforceable and human readable (so improves on ACLs)
- declarative option over scripted option reduces technical debt
- No ACL will grant you access that a data filtration has already taken out
- may need to install the 'data filtration' plugin as not installed by default yet on Tokyo version
- remember to elevate privilege to security admin first
https://www.youtube.com/watch?v=UsjbPMHVs7U
(ServiceNow )
(full video transcript:
good morning good afternoon good evening wherever you are and whenever you are welcome to creator toolbox this is the show that helps you put tools into your toolbox to be a better service now i'll get this out servicenow developer or administrator the wonders of live streaming there's no redo my name is chuck tobasi senior developer advocate at servicenow i'm going to let's do some intros going that way to earl hi everyone i am earl duque i am a servicenow developer advocate and i've been with the team for about a year now and before that i was in a lot of higher ed customers doing development work and i'll kick it off to scott hey everyone my name is scott kaufman i'm a product manager in platform security so i'm actually the product manager of data filtration i've been with servicenow for nine years going on nine years and had a customer for eight more years before that so i'm very familiar awesome and i'm laura mcmahon it's nice to meet y'all i am the newest member of the developer advocate team and i've worked at service now i just had my uh five year anniversary so very excited to still be here and very excited to hear about data filtration today and i didn't do much of an intro for me but uh i've been here since mid-2010 at servicenow as a customer for a couple of years before that and a long long resume of erroneous information before that but enough about us thank you for joining we are going to be talking about data filtration one of the new features in tokyo not just talking about it but demonstrating it as well i've built out a scenario but i want to i want to frame this up of what data filtration is it's it's it's a way to provide a declarative way uh to protect high value data because in many of the high security environments finance uh federal that kind of thing the auditors don't allow scripting and they they weren't really living up to their regulatory requirements and they were getting some compliance issues so this is our way to help address that it it provides not only access to the data like at acl but it also provides in access to what we call the subject criteria or the user's environment whether that's the ip address or what groups you belong to which obviously helps reduce scripting now data filtration complements acls it runs before the acls so you have to pass a data filtration rule if it's there before you get to all the other acl checks now whereas acls are an allow operation and scott's going to correct me if i'm saying anything out of order here this is why he's here to make sure you know that we're saying what we really heard uh in our appropriate training acls are a grant base you you allow access to something data filtration works in reverse it's a deny it's still it still has a condition part that you need to correct and we'll show this to you in the demo it has to be true to pass and then you get the filter almost like a funnel if you will what i miss these are post query operations like acls one of the questions for acls but after still possible before acls but after before query business rules yes wow there's a whole lineup of things we got to keep track of now uh we do have something on the roadmap that we'll share for later of where we're headed with that uh one of the questions in tech now was does this get rid of that annoying message at the bottom of the screen that says you know so many records are removed due to security constraints the answer is no we will still see that in the demo that we've got here let me oop i skipped ahead on my notes uh quick reminders and i'll get this in the demo as well but it does require the security admin role just like acls you are configuring security rules and who can see what so you do need that elevated security there is no admin override we'll also see that because my chuck tomasi account on this particular pdi it has admin access but you're going to see these data filtration rules apply to my account because i don't satisfy some of the conditions that we've set up it's also at the table level or the record level you can't get granular down to the fields so you're filtering out or filtering those records before that um and if you are creating these just like acls it does respect scopes so if you're creating them put them in a scope and you'll notice on ours we will be doing this in a scoped app the table list is filtered to just those scoped apps so if i'm configuring a a data filtration rule for my app my scoped app i don't have access to things like sysuser or sysuser group or task or cmdbci you won't see those it is specific to my application any further notes earl uh a clarity a clarification security admin is required to do create re uh update and deletion of records but with admin you can still read records you can read correct same same basic outline as acls turning to mr wizard scott any any words before we jump into our story here yeah i i think i'll kind of um one of the things to understand is is kind of the order of operations when it comes to accessing data um i think i'll touch on a few of the the high level points there's a whole lot of nuances that go and go on in each step but once you get past the authentication point you know the step that we go through to determine you are who you say you are um then you go and you say i want to see this data let's just say i want to see all incidents um so there's a there's a series of things that happen there's there's some pre-query activities um what most people are familiar with is you know before query business rules or query business rules however you want to call them um and then there's a lot of database stuff happening behind the scenes one of the things and this kind of this kind of goes into explaining a little bit more about the security constraints error and why we see these things in in security that's happening after the query um if you show me if you say i want to see all incidents and there's a million records in that table we fetch all million records and um because of some efficiency requirements we actually uh we actually paginate we do uh total record count calculations we do pagination calculations because those are all very resource intensive activities and uh the database does those really well if you did it after the database it would be a lot slower and performance would be not anywhere near where it is now so you imagine we have these million records that we've paginated now we've done some pagination calculations then we get handed off to other things um like access handlers and whatnot now we have data filters coming into play so we're evaluating the data we're evaluating who you are and and making a determination of whether you're allowed to see certain data subset of data or all that data in that million records and then we hand it off to acls so within that order there's some artifacts as a result of that some of the benefits is for example you mentioned um data filters runs before an acl the the value there is um because acls are an allow model it's a bunch of ores um if you have 10 acls that determine some you know criteria some scope of data access and then 11th one can come along and give access to more than the previous 10 has already you know gives you access to it's a it's a it's a downside of something that makes acl so great unfortunately but it causes some headaches for people that are trying to really secure high value data um so data filters is kind of designed to target you know high value data and know that there's no acl afterward that's going to come along and give access to it because what's been handed to acls is that has already been removed so there's nothing to add anymore um we're trying to prove upon this as well so that that order you know data filters happening after the query but before acls we actually are gonna our focus is on getting data filters to happen before the query so we've built um a query augmenter so the notion of taking a query that goes before it goes to the database we actually can intercept it now and we can decorate it with conditions um and do that very performantly we are running into some you know challenges that kind of you know slow that process but that's kind of the direction we're headed and ultimately what happens then is that chunking the the pagination of those million records if data filters and acl starts removing things after the query that that pagination becomes inaccurate and that's where you get the security constraints error and the pages unfortunately that have you know numbers that that vary if there's been records filtered out by moving data filters before the query um we are now removing data um before that pagination process occurs so if if you're only supposed to see you know a hundred thousand of the million records there's only going to be a pagination on the 100 000 records not the million records that eventually goes to acls so that kind of gives you some insight into kind of the whole you know life cycle of you know accessing data and the order of things and you know what artifacts you know are a result of that and what we're trying to change do we know when that would be expected the moving data filtration to before query uh we don't um we we're we're yeah we're so with data filters we actually are touching on an area that um we haven't we haven't needed to touch in i don't know how long i mean we're creating a new you know data control like ecl's existed since the beginning of time and so we're we're uncovering some areas where we can make improvements on that we didn't need to before um we're discovering some new things about the platform um that we can change to make improvements everywhere as because we're understanding this these operations that are happening and where the bottlenecks are um but we're we're looking at probably two releases um before we can really crack that that problem i know a lot of people will be excited when that comes out especially i mean you and me writing before query business rules are not fun you know yeah the scripting gets really complicated and it adversely affects the rest of the platform like right exporting and they're 100 scripted which is another problem i think the key word for data filtration is that it's declarative and that's why uh regular regulatory auditors are um accept declarative statements because it's so much easier to build when it's when it's easy to describe kind of yeah unexpected and there's no one-offs from the script or something like that that's i'll be excited when that comes out too yeah go ahead key design criteria was you know machine enforceable and human readable uh that's just kind of the the basis of of any sort of security that is is fully auditable and and today with acls that's not that's not the case yeah and anytime you've got a declarative option versus a scripted option take the declarative one it uh it really reduces that technical debt and your future self and your people who come after you will thank you for it i don't know how many times i've walked into someone else's application or integration or situation and going this is a lot of script to unwind as opposed to going into some records and saying i can read this condition it's very easy to figure out i'll point out in some areas when we kind of get into the demo um you know why uh you know declarative is is great and we can and where we'll start expanding and where we're focused on expanding to kind of you know accommodate more capabilities yeah but you'll see why using data filters now is still advantageous because you'll start inheriting some new stuff as we bring it along well good segue what do you say we do a quick screen share earl i'm going to bring up my scenario i can't see you guys so you gotta tell me when it's ready it's ready okay uh we've got three personas that we're gonna be using hubble tudor he's our regional director i've created an application to control stores there are 800 and what did we say 31 records in here so we've got some demo data that we can work with in various regions various countries each has an id we've got some rules that we want to apply for various sections of these ids like the lowest 1000 from 1000 to 1999 is special and secure we're going to put some data filtration rules around that uh and notice that abel and dennis don't have any roles this is a situation you may run into i've got groups but the groups don't necessarily have a role attached to them don't ask me what that means for licensing i'm not going to even try to comprehend it let alone answer it today if you want to know more about licensing my standard disclaimer applies go talk to your account account manager and they can help you now i have no groups but i do have the admin role kids don't do this at home always assign roles through groups this was only done for demo purposes so let's take a look at this here's my list of store 813 i transpose two numbers sorry 813 records some are active some aren't they all have an id they're in various countries there's even some additional information inside the record that has you know an ip address i don't know what even that means but and a code maybe that's some sort of backdoor hacker security code demo data random demo data it's not significant to anything but this will help us tell our story so in our our first scenario uh let me just also show you that i have dennis's record here dennis is logged in he sees the same 813 and abel also has the same 813. we can see that down in the bottom okay nothing special going on the acls at this point are pretty much wide open it's you can read if you have public access so that goes back to what scott was saying that no acl will grant you access that a data filtration rule has already taken out and we'll see that in a second now the first scenario is is this first thousand you see they're red they're red they're red and then they stop being red okay that means something to somebody and according to the requirement i got is admins should only be able to see those records they're special in some way i don't know what that is but only people with the admin role should see these and yes you could secure this with an acl but we're here to show you data filtration rules and then i'll show you some of the things you can do easily with data filtration that you can't do with an acl as easily or without script so let's start by going to oh should also mention we did enable a plug-in called data filtration i thought it would scott correct me if i'm wrong but i thought that was on by default for new instances or was it upgrades um no it's in in tokyo we made the decision to not make it on by default okay we wanted people to opt into it because of this this you know it's it's kind of an mvp product at this point oh sure great way to put it too i love that the um so you do have to turn it on if you choose this is a great place to do it on your personal developer instance go to developer.servicenow.com and get yourself a free pdi and then you can use it as a sandbox before you start exploring this in your subprod and broad instances in your organization so once you have it turned on you will see the data filtration menu which has data filtration records that's where we're going to be defining our rules we've also got three more here ipfilter criteria role filter criteria and group filter criteria we'll explore that in a little bit but these allow you to create uh more involved and reusable components to the data filtration records is that a fair assessment scott yeah yeah and then finally you can use a combination of these as a subject criteria and interestingly enough i discovered that subject criteria uses decision tables underneath like yeah you go to edit a certain part like the decision builder pops up that was kind of fun table exclusions is exactly what it sounds like it says i am not going to let you see or modify or do anything with these tables you can add to these tables if you like so back to data filtration records let's create a new one what's the first thing i need to do earl okay remember you must have security admin so you got to escalate yourself yes and we won't turn this into a drinking game how many times does chuck forget there we go now i have a new button if you don't see the new button you don't have security admin that's your clue so rule number one pick your table and it will happen to everyone it will happen it has happens everybody anyway for acls notice that i only have one table if you are going where are my other tables if you say show notice that we have a table choice script that does what it's supposed to and says this is what i'm going to show you in this table field so i'm going to pick my store table and it is active just like most records in the system you can still keep it just make it deactive try not to delete things cascading means if you set one of these data filtration rules on a parent class such as task this will apply to all the records in the extended tables incident change problem i would i use that one because it's easier than going through several cmdb examples let's give it a nice friendly description special low numbers for admins only uh scott i also discovered that description is only 40 characters so it's not even a short description it's a shorter description oh wow okay we might want to look at that i think that's in a couple of places it might be on the subject criteria as well all right i had a lot of fun going through everything we discovered lots of stuff this is you know patch zero this is the first release of the first release of the mvp okay we've got data filter these are the two important sections here data filter and subject condition if it were me designing this form i probably would have reversed these because i tend to think of them in the reverse order like if this then that but you do what you want to do i think that the idea here was we wanted to continue some familiarity with acls which is really focused on the data first ah okay i can but behind that you might see some you might see some design you know user experience changes around this because we are very concerned about fragmentation of features so there is some consolidation on the horizon here as well nice so let's do our condition here we want to only show where the id is between uh this is something you definitely would have to script for in an acl 1000 and 1999 right easy enough i'm getting a little concerned so i save off and save early that's what i want to show if you meet these conditions before you jump to that go back to the data filter absolutely two things here one is that the going back to the cascading as it relates to the conditions so if you're on the task table for example and you want to cascade this down to all table extensions it's important to understand that the the conditions available to you are only on the table that you selected so you're not going to see incident specific conditions even though you're cascading this in down into the to potentially into the incident table so the the ids are of the table you selected in the in the actual table field good point um so if you do need to have an incident specific field as part of your condition you would need to create a data filter at the incident table level i see it the drop down doesn't have the extended fields option then i'm assuming right it has it has show related fields even though oh no i guess it doesn't on this one yeah just on the on a regular on the on different interfaces it's the show extended fields to be able to do that additionally but in this in data filtration you would want to build it in a different record yeah so like cascading in order to do that it would have to collect all the fields across all the tables and then you start causing potential issues where an incident specific field is not in the task table and so how does it interpret that and enforce it properly the whole point is trying to be declarative and if you have a billion filters on one data filtration maybe you need to re-look at how you do data possibly yeah the other thing to point out is similar to acls that we have that preview button next to data condition um and that's an easy way to see what records match the criteria and then also um uh it will allow you to click on the actual record match count and view the preview those records i was wondering okay condition builder equals v2 attribute is what puts up the nice preview button instead of the little circley circley thing gotcha yeah all right you you you definition fields when you use these three [Laughter] thank you so many times we've created condition fields without readable equals true and i'm so annoyed when i see a code yeah in the list so now you can actually see the records themselves by clicking on it it's just like acls but that's awesome yeah i mean if we do that you'll see right there there's our data condition it actually reads instead of you know this encoded query let's go back in there and set our subject condition so step two is what do i need in order to see that according to what i said before the user needs to have the admin role so i use the chooser here again and i've got the ability to look at network criteria could be something i've built or could be yeah no that has to be something i built in that network criteria list or group or role or subject criteria subject criteria subject group and subject role apply to what i am i am the subject in this so let's make it easy and say subject role is admin easy enough this is now live what do you say we go look favorites ungrouped stores i am admin i see everything below one thousand hooray there's two thousand six at the bottom of the list below two thousand the follow two thousand thank you in the one time if it starts with a one i win now let's refresh dennis's list and haha can't see those and he starts counting at 2006 and then from there on it's pretty much what you would expect sounds a lot honestly that's a lot easier to configure than an acl and abel should be the same way right minus 50 minus 48 there's the other two yeah that wasn't too bad pretty straightforward what do you say we do another one okay i have another requirement uh only show these ams records like this one if you are in the ams support group okay again you would have to resort to scripting with an acl for something like this we're going to do it of course with data filtration so i have a favorite for my rules let's create a new one again on the store table well dynamic filters would be able to do it in the condition builder for nacl uh yes you could you're right you're right so our data filter is going to be store ams support that's what i'm going to show you and that would give me 19 records or take away 19 records from my list and the subject condition is going to be this time subject group is ams support now i did hear a question on another video that said can i do show me all of my groups and this is not a dynamic filter it just has an is operator and this does have a dynamic filter which you could do here so i'd be saying if i was in the ams group show me all of the records for all of my groups which might be a little too broad so you got to think closely about that recognizing that let's go back to my original one just because it will stay on script this doesn't have any more conditions for the groups than just ears there isn't is one of or is dynamic yet should i say scott yeah possibly let me touch on that for sure um it's something to kind of peel apart a little bit and understand um so you have to imagine that uh we are building something that is uh potentially being called in every transaction yeah as you add more data filters we're talking about high transaction um part of the the system the when we start adding the the is dynamic right there's it's basically you know mini javascript call there is an extra operation from the subject side so if you think about it from a query standpoint so our goal is trying to get this to be done before the query [Music] as far as the dynamic variables for data that can be that's basically translated into a modified sql query in order to determine whether or not you satisfy the criteria you know the subject conditions if we started adding dynamic capabilities we would actually be running scripts and potentially database calls before we did our main database calls so we're adding we're adding extra steps in inexpensive steps so we have to be careful in how we evaluate you you know you as a person to determine your level of access um if we touch on kind of what the the area that we're going to be going more on and i can i can go into this a little bit later when it makes more sense but um the the uh subject conditions are going to grow we we just maintain security role group and ip address and actually if any of you are familiar with adaptive authentication which is also something fairly new to us we're sharing the same code so the same rules that you define to determine authentication adaptive authentication rules we you can carry that over into your data conditions as well oh nice yeah so we so we we plan on um getting more into the subject and environment attributes and and allow for kind of customization and configurability of that um and i can talk about that a little bit in a little bit if if we want to but that's where we want to grow beyond just role group and network adding network is actually a great thing because that doesn't exist today in the acl so right um that's key that's that's that's this you know we're sharing code we're eventually going to bring all this all the way into the acl as well so a lot of the stuff the declarative stuff we're going to be adding to acls um and so you know that that's that's kind of where we're where we're focused on growing all right and i am am i safe to assume you can't do you know javascript colon something in here um well there's a matter of if you can or you should which i'm not sure which is good valid good answer just because you can do a thing doesn't mean you should do a thing right i wouldn't be married if that happened okay let's see how it worked so theoretically chuck is not part of ams support group according to that slide earlier but if i look at stores ungrouped and actually you know what i have a shortcut just for that so grouped by support group i don't have any ams let's see what dennis gets by support group dennis is also not in he's in the emit mes support group and if we go over and look at our old friend abel tutor not history hey where's my favorites anyway group by support group he does have ams because he's in that group so proof that we can easily create a data filtration record for that specific group i have one more but i want to show you something first if you're ever questioning why don't i see this you can do debug all security up our favorite debugger window i'm going to allow and when i refresh this list see if i can get that back there we go i can search on filter right and hit this and determine exactly which ones applied and which ones did not so data filtration applied subject criteria not met support group blah blah blah data filtration not applied subject criteria met the id is between so i can tell exactly why did this record show up or not is based on these records so use the uh security debugging to your advantage thank you for making sure that's included with its own messages by the way you're welcome we wanted to uh yeah do better than we've done in the past [Laughter] you're driving a good point which is which is also going to be more meaningful in the future right so as we you know you see right now that oh there's i'm missing some data because i don't satisfy some criteria um and so that prompts you to you know go into the debug in the future when we are able to move this at a quick you know before the query as part of the query um those those uh those notices will go away and um everyone will be seeing the data that they presumed are allowed to see so even more so um the debugger is going to come into play as kind of like your go-to rather than just seeing the error itself or the message itself saying okay what's being hidden and why and then go to the debugger oh good point good point okay i'm going to do a different approach to that last example where i said the ams support group gets to see the ams records we're going to satisfy dennis so you can see only dennis can see the email support records but i'm going to do it in a slightly different way using something called support criteria so underneath oh what was i typing yesterday paf i like i like working on the middle of the word for uniqueness we're going to create a support criteria that we can then use in our data filtration record this is a small example of where you can go with this so we are going to create a new called i like to just call these i'm going to prefix this because ams or emea support is going to come in all over the place and this will help me tell them apart i mean by no means condoning a best practice here i only got confused yesterday so maybe this turns into a best practice description hey this description field is bigger i like this so save that and down here i've got two more tabs and this is where i may lean on scott a little bit to help us understand what the criteria inputs are so um as as you mentioned before this is go this is using uh decision tables uh and so the the prerequisite for decision tables is to define what inputs you want as part of that okay so it's essentially just yeah adding what criteria what of the three that you know we're able to evaluate what you want to add to this um and it's just kind of a nuance with using decision tables all right and this is going to these three correspond to these three correct so it's going to be giving me records in these other tables implicitly yeah the the the other yeah those those other um links in the menu are probably um you know just extraneous they're just there you probably don't need to interact so much with those but we kind of guide you through the process oh hardcore people love them true that's probably why they're still there now i could create any one of those ipfilter criteria to say hey i want this ip range or cider address or whatever uh you can make these as simple or as complex as you want this is a group filter criteria for email support again reusable components in here and i'm going to assign it the group store support that is that piece of it and notice that it's currently says used false now this is the cool part these records know where they're referenced so you can't delete the wrong thing first if i came and tried to delete the support criteria it would go i i'm sorry if you try to create the group filter criteria it would go hang on this is being used somewhere you don't want to yank the carpet out from under this or or saw the branch off that you're sitting on use whatever metaphor you like for disaster happening here the system will actually tell you that you can't delete this record because it's used in a a subject criteria which i think is really cool though something i did notice is you can actually still set that field as active or false even though it should be it is automatically determined by the criteria conditions so maybe in the future we can make that a read-only field oh yeah protect it whoop yep you're right good call earl mvp product yeah and we we want to do better in this area um make it a bit more intuitive because either we we realize it's not but um that's just kind of how it has the components are necessary uh we want to do better though and scott if i add more criteria inputs let's say i say you've got to be in this group on this ip range etc you have this role are those ended or ored together uh that that's defined in the conditions so all you're doing right now in this first tab is just bringing uh listing you're bringing all the available inputs that you want to evaluate in the condition okay so it's it's kind of like atf in a way where you're setting things up that you can use later or flow designer yeah so think of you know in our condition builder there's the left side and then the right side right so you're just basically uh creating the the list of left options right now let's go do that let's let's show these people how that works so label this is my subject criteria creation for me support you see why i'm starting to do that and in here oh look yeah i've got the different filter criteria uh earl and i were wondering what sis id and keywords are doing in here that's a good question okay generic to the condition field i don't know we were conjecturing about those two for a little bit last night no attributes on this one so maybe maybe if you use v2 they go away or something i don't know so what we're asking is when this decision table evaluates our decision we want it to come back as true now you could invert this which says you are not part of that group is that correct yeah okay easy enough pretty straightforward well yes uh i love that yes and i'm i'm doing mental gymnastics right now to to verify that answer okay we can we can easily flip that and see who can see what records in a minute but if you want to see the decision table up comes decision builder our old friend thank you julia perlis and her team for creating this it's not that hard to glance at it and go what's happening here uh if you're part of this group it's true okay let's go back i didn't make any changes let's go back again okay show the first tab again first tab is now true it says you have been used so if i combine three of these and i like that if if you said i've got a group one and of course you can always personalize the list to say you know what's the display name what's the filter criteria blah blah blah to make it a little more readable you go oh my filter criteria for email support is used but i better go back to the criteria conditions because the one with the role or the one with the ip address wasn't included in that condition so it gives you a little visual before hopping over here is this right let me open that up and see if it actually says what i wanted to say so in the in the midst of all this i think it's important not to lose context of what we're trying to achieve um and and the value of it so if you think about um you think about creating um a number of data filters that have some let's just call them compound conditions so you have to be a member of a couple different groups or be from this role or from the corporate network um and you want to repeat those that that those compound conditions and and have consistency across multiple data filters this is this is how you would go about that um almost almost like in it to building a persona that's using a combination role group and network right oh my thunder man actually your thunder have added yeah the reusability and i'll show you real quickly now that we've got the subject criteria defined that persona that you know scott alluded to i'm going to go to like the data filtration and create a new filtration record to enact that or enroll that store let's see uh email for email or it's like a protest statement or something beta filtration similar to before we're going to say the support group is store emea that's what you get to see if you satisfy this condition now this is where subject criteria comes in i'm going to use that other subject now i have to start typing it so fortunately i started with an sc for subject criteria and there we go so in theory if this worked fingers crossed i go to chuck's list chuck is not in ams or email so that's working dennis darren i don't know i keep calling dennis because it's darren ennis or enos inus nice something what's his name darren nice okay the answer is yes to emea no to ams and if we go over to abel and refresh the list you know you can refresh the list just by clicking all aha things have been taken away no no nobody in the first thousand because that rule is applying and group by support group says no email should we flip it around just to test your theory scott okay we're off script just letting everybody know that's like we have a really he's like we have a really good batting average we don't want to change it up now right this is we're developing into the criteria and we flip this to false so this would be a way to exclude data from people not in that group let's try it again favorites grouped by support group uh chuck doesn't see it interesting refresh the list i'm not sure what happened to my refresher here still don't see it for dennis and still don't see it here now this is where there's like there's a couple of things we could talk about now chuck based off of what we found last night we're going to just try clearing the cache let's see what impact that has we could also try logging out logging in but i'm not ready to go there yet okay cache is cleared chuck refreshes does chuck see records we have to rebuild that cash which is why this is taking a couple extra seconds this is when you are glad you have a cash no i noticed another decoration i don't know if this was san diego or tokyo you see up here where you got these new little icons here you can see what is grouped by so many times i've turned on grouping and i don't remember which column was on totally this is nice i'm thank you for whoever did that i think this is a great time to mention some of the uh quirks that we ran into um in our testing check so when we were figuring out uh and setting up this demo some some things came up where we realized two things one uh impersonation the reason why chuck is showing them in its own session windows right now instead of impersonating a user to demonstrate the records is because what we've noticed is that the the essentially the criteria that is hitting for the data filtration persist to the original person that's logged in even if you impersonate to somebody else so impersonation is not a great thing to use while trying to test for these records in data filtration so i see scott writing things down i think so that's um uh i see him on the side um the other thing is uh right now chuck is in a scoped application and we saw some irregularities while being in a scoped application when it came to the subject criteria yeah subject criteria specifically um so if we're running into interesting things here we don't know necessarily if it's a scoped issue or if it's a bug for the entire the actual feature itself but um when it comes to subject criteria we knew that it worked in global but not in a scoped application yeah for scoping um there is we can definitely do better in this area um we obviously as you can tell we're we support scopes but then you when you add like scoped administration or scope security in into the picture it complicates it significantly um so we've we've identified some areas where we can definitely improve and um have more expected behavior around that um and also kind of you know do bet be better than what scopes allow us to do one more reminder is some of you may know if you use a glide record query you can navigate around acls and if i'm correct you can you can do that today with glide record on data filtration but you cannot do it i mean the glide record secure honors the acls and the data filtration rules but the goal is no more no more working around with glide record they are what they are especially if they're pre-query you got no choice yeah that's something good to kind of dive in a little bit on um you know as we as we kind of thought of data filters you know charter is to kind of protect high value data one of the one of the things that we identified an area that we identified that we wanted to try and improve upon was um i feel like a better term kind of centralizing security so taking taking the choice out of the hands of the developer of whether or not security is enforced or not so we kind of took a position of let's let's not make data filters bypassable that led us that led us down a very long rabbit hole that ultimately was a dead end not because we can't solve the problem but data filters it wasn't data filters job or responsibility to solve what we ended up finding is a very high risk of regression so if you think about it you know all of our what we ship out you know from in our baseline code we have some expectations around what level of access certain processes have and you can't get around you can't get around the need for a process to be able to bypass security you know the system itself needs access to data in order to process the data so you have to have some level of you know bypass ability of that security and today that mechanism is glide record versus glad record secure and unfortunately both of those apis are publicly available as in script so you can essentially create scripts that bypass security um this is nothing new um this is just that's just the way it is so the notion of trying to change the um the point where data filters was enforced i mean we wanted to go down to the db um the db query layer um but what we found is that we were um we were creating an environment of a very high risk of regression so like you should you know there's no data filters shipped out of the box but suddenly you start creating you know data filters and applications start misbehaving because they weren't tested um you know with with security in place maybe that that that that application needs acts full access to data but you were creating a data filter that was removing access and and you'd start seeing anomalies so we we didn't make it made a determination that um we need to solve that problem more at the plat you know general platform layer rather than within data filters itself um so we've kind of resolved to re you know reevaluating our approach to that problem one other quirk that we came up with is you can see abel here doesn't have access to any of the 1000 records but if i were to drill into one of those records and say email the link to abel away we go there we go had to remind so that's something to consider as well i don't know if if that's on the road map as well scott but yeah we actually have a defect uh we're working at hurricane as a defect at the moment good so if you're watching this in the future and it doesn't work for you you know why all right that's um you know unless we wanted to go down the global path for a quick demo i think looking at the clock we better get it back on track but we're aware of these features this is patch zero uh i got nothing else to demo here earl i think we can probably run through some of the questions that were asked inside of the chat during the demo and just make sure we covered everything oh wait i do want to show one more thing hold on okay work filtration uh if you try to delete that group criteria i said it's got a smart delete and i go i'm going to be sneaky and try and delete it it says no you can't it's already being used click here to review and it brings you right to you know where you need to go okay yes let's see some questions little things like that save a lot of headaches thanks for making sure that's there that was fun give me a little warning first chuck dates i'm gonna fix that too a few questions popped them in chat um some things we a lot of these we did answer on air um just by coincidence but i just want to make sure that we covered everything um stefan had a question about phil data filtration and how we're expecting admins to be able to uh work on records that are being reported to us if there's no admin override button and stuff like that and one of the key things is debug security still happens um so you'll there's key messages in there saying hey data filtration is getting rid of this record for this person for for you that's why you can't see it and that's why there's a bug um so you need to go look at data filtration or if it's not that go to acls then um but one of the other things is uh one thing we did we mentioned uh the url bypass chuck yes so that when when you went into the specific record right okay um so there is a stuff that we uh noticed about that that um uh it's mainly a table-less kind of thing known defect yeah uh we had a lot of questions for clarity about acls working with data filtration and just for clarity uh they they work to complement each other so remember they're both post query uh data filtration works first and then passes those lists of records over to acls to run so they work together and they always work together you can't have one or the other if you turn on the data filtration uh what else is in there uh another clarification of about does this get rid of the security prevented showing the rows that's part of post query so because data filtration is also post query it doesn't get rid of that what you're looking for is a before query business rule right now but as uh scott has mentioned in the future we hope to move data filtration or give the option to have that run before query which would solve that problem also um nitish gave a good summary of what happens and then i clarified it with uh hey um it's more clear if you say where the query was so that you know what happens before querying what happens after query uh to be a good picture of the the workflow that's happening on records behind what else has been asked there was the one about cascading too yeah so earlier in the episode um we talked about cascading and scott gave great insight on uh gotchas to make sure to pay attention for about the condition builder for a data filtration record and to make sure to build the if you have a cascading field that you need to be addressing you need to you would want to uh because of the declarative way of how this works is to build a new data filtration record on that table so a lot of cool clarifications a lot of um questions in the chat which is great because we wanna that's how we get all the information out there and how we get uh all the information to scott too this is great this is why we love having a product manager on our um on our shows uh but yeah i think that was a lot of questions a lot of the things a lot of the chat was a lot of clarification on things um that we did answering the show so i won't go over um a lot of these questions again uh but did you notice anything else in the chat lauren um did you go through all your notes chuck yep earl point up with your left finger this is part of the tokyo content you can tell because we have this cool badge we are in the midst of the tokyo season you can find out everything that's out there or going to be out there on our tokyo calendar i think we may need to update a few links because it's been a couple of weeks now we've got content rolling out creator toolbox live coding happy hour breakpoint podcasts blog entries lots of stuff coming in devlink dot sn slash tokyo earl's got the banner right there well-oiled machine any other announcements earl we are we have a uh do you want to talk ab challenge at all build with aes challenge yes lauren heck yeah so i believe it was about three or two or three weeks ago we kicked off the build with aes challenge uh all the information about the challenge is listed on our blog and also on the community but essentially if you can write a short blog post or film a short video on anything that you've built with aes why you liked it what you liked about the experience we will send you a free shirt it is very similar to this shirt that was oh that went like lickety-split out of stock at creatorcon so if you missed out on the shirt there uh this is an awesome opportunity to get a very good uh dupe without the uh little creatorcon specific logo but the whole shirt is the same i'll be the one to ask what does aes stand for app engine studio thank you so much i appreciate that [Laughter] well i know i just want to make sure we say just because um natasha asked about it uh he's saying that if the if you have trouble with the dev links uh just go to developer.servicenow.com and check out uh all of our resources um on the blog and you'll be able to see the tokyo content uh breakpoint content this shows content our friday live coding happy hour content all that's there on developer.servicenow.com and then click on blog plus you'll see all all the other resources that you have available to you as a developer so check it out all right i just tested it we did have a technical issue last week because rit people changed something on the back end that we had to track down so it was broken for about 12 hours but hopefully that's all cleared up i just tested devlin slash tokyo is working but you can also find that on the blog as well thank you earl thank you lauren for joining and definitely thank you scott for the depth of knowledge on this product absolutely thanks guys definitely another couple degrees higher than where we could have taken it uh had i had a little more time i would have made nicer slides until next time everybody take care stay safe and we will see you soon )
Comments
Post a Comment