Patent:Interactingwithvisualcodeswithinmessagingsystem
PublicationNumber:20240281121
PublicationDate:2024-08-22
Assignee:SnapInc
Abstract
Aspectsofthepresentdisclosureinvolveasystemcomprisingacomputer-readablestoragemediumstoringaprogramandmethodforinteractingwithvisualcodeswithinamessagingsystem.Theprogramandmethodprovidefordisplaying,byamessagingapplication,capturedimagedatacomprisingavisualcode,thevisualcodeincludingacustomgraphicandbeingdecodabletoaccessafirstfeatureofthemessagingapplication;receivinguserinputselectingthevisualcode;displayinganupdatedversionofthecustomgraphic;providingananimationwhichdepictstheupdatedversionofthecustomgraphicasmovingfromthevisualcodetoaninterfaceelementcomprisingagroupoficons,eachiconwithinthegroupoficonsbeinguser-selectabletoaccessarespectivesecondfeatureofthemessagingapplication;andupdatingthegroupoficonstoincludeanadditionaliconwhichisuser-selectabletoaccessthefirstfeatureofthemessagingapplication.
Claims
Whatisclaimedis:
1.Amethod,comprising:displaying,byanapplicationrunningonaclientdevice,capturedimagedatacomprisingavisualcode,thevisualcodeincludingacustomgraphicandbeingdecodabletoaccessafirstfeatureoftheapplication;receiving,bytheapplication,userinputselectingthevisualcode;andinresponsetoreceivingtheuserinput,updatingagroupoficons,eachiconwithinthegroupoficonsbeinguser-selectabletoaccessarespectivesecondfeatureoftheapplication,toincludeanadditionaliconwhichisuser-selectabletoaccessthefirstfeatureoftheapplication,whereinthefirstfeatureandeachoftherespectivesecondfeaturescorrespondtoarespectiveaugmentedrealitycontentitemforapplyingtothecapturedimagedata;downloading,bytheclientdeviceandfromaserver,codeforafirstaugmentedrealitycontentitemcorrespondingtothefirstfeature;andactivating,basedondownloadingthecode,thefirstaugmentedrealitycontentitemwithrespecttothecapturedimagedata.
2.Themethodofclaim1,furthercomprising:displaying,inresponsetoreceivingtheuserinput,anupdatedversionofthecustomgraphic.
3.Themethodofclaim2,furthercomprising:providingananimationwhichdepictstheupdatedversionofthecustomgraphicasmovingfromthevisualcodetoaninterfaceelementcomprisingthegroupoficons.
4.Themethodofclaim3,whereintheinterfaceelementisacarouselinterfacefornavigatingthegroupoficons.
5.Themethodofclaim2,whereintheupdatedversionofthecustomgraphiccorrespondstoapredefinedstaticimage.
6.Themethodofclaim2,whereintheupdatedversionofthecustomgraphiccorrespondstoapredefinedanimation.
7.Themethodofclaim1,whereinthevisualcodefurtherincludesauniqueconfigurationofmarkingswhichisdecodabletoaccesstothefirstfeature.
8.Themethodofclaim1,whereindownloadingthecodeisinresponsetoseconduserinputselectingtheadditionaliconwithinthegroupoficons.
9.Themethodofclaim1,whereindownloadingthecodeisinresponsetoreceivingtheuserinput,andwhereinactivatingthefirstaugmentedrealitycontentitemisinresponsetoreceivingseconduserinputselectingtheadditionaliconwithinthegroupoficons.
10.Adevicecomprising:atleastoneprocessor;andamemorystoringinstructionsthat,whenexecutedbytheatleastoneprocessor,configuretheatleastoneprocessortoperformoperationscomprising:displaying,byanapplicationrunningonthedevice,capturedimagedatacomprisingavisualcode,thevisualcodeincludingacustomgraphicandbeingdecodabletoaccessafirstfeatureoftheapplication;receiving,bytheapplication,userinputselectingthevisualcode;andinresponsetoreceivingtheuserinput,updatingagroupoficons,eachiconwithinthegroupoficonsbeinguser-selectabletoaccessarespectivesecondfeatureoftheapplication,toincludeanadditionaliconwhichisuser-selectabletoaccessthefirstfeatureoftheapplication,whereinthefirstfeatureandeachoftherespectivesecondfeaturescorrespondtoarespectiveaugmentedrealitycontentitemforapplyingtothecapturedimagedata;downloading,fromaserver,codeforafirstaugmentedrealitycontentitemcorrespondingtothefirstfeature;andactivating,basedondownloadingthecode,thefirstaugmentedrealitycontentitemwithrespecttothecapturedimagedata.
11.Thedeviceofclaim10,theoperationsfurthercomprising:displaying,inresponsetoreceivingtheuserinput,anupdatedversionofthecustomgraphic.
12.Thedeviceofclaim11,theoperationsfurthercomprising:providingananimationwhichdepictstheupdatedversionofthecustomgraphicasmovingfromthevisualcodetoaninterfaceelementcomprisingthegroupoficons.
13.Thedeviceofclaim12,whereintheinterfaceelementisacarouselinterfacefornavigatingthegroupoficons.
14.Thedeviceofclaim11,whereintheupdatedversionofthecustomgraphiccorrespondstoapredefinedstaticimage.
15.Thedeviceofclaim11,whereintheupdatedversionofthecustomgraphiccorrespondstoapredefinedanimation.
16.Thedeviceofclaim10,whereinthevisualcodefurtherincludesauniqueconfigurationofmarkingswhichisdecodabletoaccesstothefirstfeature.
17.Thedeviceofclaim10,whereindownloadingthecodeisinresponsetoseconduserinputselectingtheadditionaliconwithinthegroupoficons.
18.Thedeviceofclaim10,whereindownloadingthecodeisinresponsetoreceivingtheuserinput,andwhereinactivatingthefirstaugmentedrealitycontentitemisinresponsetoreceivingseconduserinputselectingtheadditionaliconwithinthegroupoficons.
20.Thenon-transitorycomputer-readablestoragemediumofclaim19,theoperationsfurthercomprising:displaying,inresponsetoreceivingtheuserinput,anupdatedversionofthecustomgraphic.
Description
CROSS-REFERENCETORELATEDAPPLICATION
ThisapplicationisacontinuationofU.S.patentapplicationSer.No.17/820,674,filedAug.18,2022,whichisincorporatedbyreferencehereininitsentirety.
TECHNICALFIELD
Thepresentdisclosurerelatesgenerallytoamessagingsystem,includinginteractingwithvisualcodesforaccessingfeatureswithinamessagingsystem.
BACKGROUND
Messagingsystemsprovidefortheexchangeofmessagecontentbetweenusers.Forexample,amessagingsystemallowsausertoexchangemessagecontent(e.g.,text,images)withoneormoreotherusers.
BRIEFDESCRIPTIONOFTHESEVERALVIEWSOFTHEDRAWINGS
Inthedrawings,whicharenotnecessarilydrawntoscale,likenumeralsmaydescribesimilarcomponentsindifferentviews.Toeasilyidentifythediscussionofanyparticularelementoract,themostsignificantdigitordigitsinareferencenumberrefertothefigurenumberinwhichthatelementisfirstintroduced.Somenonlimitingexamplesareillustratedinthefiguresoftheaccompanyingdrawingsinwhich:
FIG.1isadiagrammaticrepresentationofanetworkedenvironmentinwhichthepresentdisclosuremaybedeployed,inaccordancewithsomeexamples.
FIG.2isadiagrammaticrepresentationofamessagingsystem,inaccordancewithsomeexamples,thathasbothclient-sideandserver-sidefunctionality.
FIG.3isadiagrammaticrepresentationofadatastructureasmaintainedinadatabase,inaccordancewithsomeexamples.
FIG.4isadiagrammaticrepresentationofamessage,inaccordancewithsomeexamples.
FIG.5isaninteractiondiagramillustratingaprocessforinteractingwithvisualcodesforaccessingfeatureswithinamessagingsystem,inaccordancewithsomeexamples.
FIG.6illustratesauserinterfaceinwhichavisualcodeisdisplayedwithincapturedimagedata,inaccordancewithsomeexampleembodiments.
FIG.7illustratesauserinterfaceinacasewheretheuserselectsavisualcodetoactivateanaugmentedrealitycontentitem,inaccordancewithsomeexampleembodiments.
FIG.8illustratesauserinterfaceinwhichacustomgraphicofavisualcodeisupdatedinresponsetouserselectionofthevisualcode,inaccordancewithsomeexampleembodiments.
FIG.9illustratesauserinterfaceinwhichanupdatedcustomgraphicisanimatedtomovetowardandbeinsertedwithinacarouselinterface,inaccordancewithsomeexampleembodiments.
FIGS.10A-10Cillustrateuserinterfacescorrespondingtodifferentmannersinwhichanaugmentedrealitycontentitemcorrespondingtoavisualcodeisdownloadedandactivated,inaccordancewithsomeexampleembodiments.
FIG.11isaflowchartillustratingaprocessforinteractingwithvisualcodesforaccessingfeatureswithinamessagingsystem,inaccordancewithsomeexamples.
FIG.12isaflowchartforanaccess-limitingprocess,inaccordancewithsomeexamples.
FIG.13isadiagrammaticrepresentationofamachineintheformofacomputersystemwithinwhichasetofinstructionsmaybeexecutedforcausingthemachinetoperformanyoneormoreofthemethodologiesdiscussedherein,inaccordancewithsomeexamples.
FIG.14isablockdiagramshowingasoftwarearchitecturewithinwhichexamplesmaybeimplemented.
DETAILEDDESCRIPTION
Amessagingsystemtypicallyallowuserstoexchangemediacontentitems(e.g.,messages,imagesand/orvideo)withoneanotherinamessagethread.Asdescribedherein,amessagingsystemmayprovidevisualcodesQRcode,1Barcode,DataMatrix,Azteccodewhicharescannableforaccessingparticularfeaturesofthemessagingsystem.
Thedisclosedembodimentsrelatetoamessagingsystemforinteractingwithvisualcodestoaccessfeaturesofthemessagingsystem.Inothersystems,afterselectingthevisualcode,auserispresentedwithaseparatemessagebox(e.g.,anoverlay)thatpromptstheusertodownloadand/oractivatethefeature.Thismessageboxmaybedistracting,asitswitchestheuser'sattentionawayfromthecurrentinterfacerelatedtothefeature.
Thedisclosedembodimentsprovide,inresponsetouserselectionofavisualcode,amessagingclienttoupdateacustomgraphicwithinthevisualcode.Themessagingclientanimatesmovementofthecustomgraphicfromthevisualcodeareaofthedisplaytoacarouselinterface.Thecarouselinterfaceincludesuser-selectableiconsrepresentingrespectivefeaturesofthemessagingsystem.Themessagingclientupdatesthecarouselinterfacetoincludeanadditionaliconcorrespondingtothevisualcode,anddownloadsthecode(e.g.,assets)foraccessingthecorrespondingfeature.Thedownloadandactivationofthefeaturemaybeautomatic(e.g.,withoutfurtherpromptstotheuser),toreducetheamountofinteractionrequiredfromenduser.
FIG.1isablockdiagramshowinganexamplemessagingsystem100forexchangingdata(e.g.,messagesandassociatedcontent)overanetwork.Themessagingsystem100includesmultipleinstancesofaclientdevice102,eachofwhichhostsanumberofapplications,includingamessagingclient104andotherapplications106.Eachmessagingclient104iscommunicativelycoupledtootherinstancesofthemessagingclient104(e.g.,hostedonrespectiveotherclientdevices102),amessagingserversystem108andthird-partyservers110viaanetwork112(e.g.,theInternet).Amessagingclient104canalsocommunicatewithlocally-hostedapplications106usingApplicationsProgramInterfaces(APIs).
Amessagingclient104isabletocommunicateandexchangedatawithothermessagingclients104andwiththemessagingserversystem108viathenetwork112.Thedataexchangedbetweenmessagingclients104,andbetweenamessagingclient104andthemessagingserversystem108,includesfunctions(e.g.,commandstoinvokefunctions)aswellaspayloaddata(e.g.,text,audio,videoorothermultimediadata).Insomeembodiments,themessagingclient104activatesacameraoftheclientdevice102(e.g.,uponstartupofthemessagingclient104).
Themessagingserversystem108providesserver-sidefunctionalityviathenetwork112toaparticularmessagingclient104.Whilecertainfunctionsofthemessagingsystem100aredescribedhereinasbeingperformedbyeitheramessagingclient104orbythemessagingserversystem108,thelocationofcertainfunctionalityeitherwithinthemessagingclient104orthemessagingserversystem108maybeadesignchoice.Forexample,itmaybetechnicallypreferabletoinitiallydeploycertaintechnologyandfunctionalitywithinthemessagingserversystem108buttolatermigratethistechnologyandfunctionalitytothemessagingclient104whereaclientdevice102hassufficientprocessingcapacity.
Themessagingserversystem108supportsvariousservicesandoperationsthatareprovidedtothemessagingclient104.Suchoperationsincludetransmittingdatato,receivingdatafrom,andprocessingdatageneratedbythemessagingclient104.Thisdatamayincludemessagecontent,clientdeviceinformation,geolocationinformation,mediaaugmentationandoverlays,messagecontentpersistenceconditions,socialnetworkinformation,andliveeventinformation,asexamples.Dataexchangeswithinthemessagingsystem100areinvokedandcontrolledthroughfunctionsavailableviauserinterfaces(UIs)ofthemessagingclient104.
Turningnowspecificallytothemessagingserversystem108,anApplicationProgramInterface(API)server116iscoupledto,andprovidesaprogrammaticinterfaceto,applicationservers114.Theapplicationservers114arecommunicativelycoupledtoadatabaseserver120,whichfacilitatesaccesstoadatabase126thatstoresdataassociatedwithmessagesprocessedbytheapplicationservers114.Similarly,awebserver128iscoupledtotheapplicationservers114,andprovidesweb-basedinterfacestotheapplicationservers114.Tothisend,thewebserver128processesincomingnetworkrequestsovertheHypertextTransferProtocol(HTTP)andseveralotherrelatedprotocols.
Theapplicationservers114hostanumberofserverapplicationsandsubsystems,includingforexampleamessagingserver118,animageprocessingserver122,andasocialnetworkserver124.Themessagingserver118implementsanumberofmessageprocessingtechnologiesandfunctions,particularlyrelatedtotheaggregationandotherprocessingofcontent(e.g.,textualandmultimediacontent)includedinmessagesreceivedfrommultipleinstancesofthemessagingclient104.Aswillbedescribedinfurtherdetail,thetextandmediacontentfrommultiplesourcesmaybeaggregatedintocollectionsofcontent(e.g.,calledstoriesorgalleries).Thesecollectionsarethenmadeavailabletothemessagingclient104.Otherprocessorandmemoryintensiveprocessingofdatamayalsobeperformedserver-sidebythemessagingserver118,inviewofthehardwarerequirementsforsuchprocessing.
Theapplicationservers114alsoincludeanimageprocessingserver122thatisdedicatedtoperformingvariousimageprocessingoperations,typicallywithrespecttoimagesorvideowithinthepayloadofamessagesentfromorreceivedatthemessagingserver118.
Thesocialnetworkserver124supportsvarioussocialnetworkingfunctionsandservicesandmakesthesefunctionsandservicesavailabletothemessagingserver118.Tothisend,thesocialnetworkserver124maintainsandaccessesanentitygraph304(asshowninFIG.3)withinthedatabase126.Examplesoffunctionsandservicessupportedbythesocialnetworkserver124includetheidentificationofotherusersofthemessagingsystem100withwhichaparticularuserhasrelationshipsoris“following,”andalsotheidentificationofotherentitiesandinterestsofaparticularuser.
Inresponsetoreceivingauserselectionoftheoptiontolaunchoraccessfeaturesoftheexternalresource,themessagingclient104determineswhethertheselectedexternalresourceisaweb-basedexternalresourceoralocally-installedapplication106.Insomecases,applications106thatarelocallyinstalledontheclientdevice102canbelaunchedindependentlyofandseparatelyfromthemessagingclient104,suchasbyselectinganicon,correspondingtotheapplication106,onahomescreenoftheclientdevice102.Small-scaleversionsofsuchapplicationscanbelaunchedoraccessedviathemessagingclient104and,insomeexamples,noorlimitedportionsofthesmall-scaleapplicationcanbeaccessedoutsideofthemessagingclient104.Thesmall-scaleapplicationcanbelaunchedbythemessagingclient104receiving,fromathird-partyserver110forexample,amarkup-languagedocumentassociatedwiththesmall-scaleapplicationandprocessingsuchadocument.
Inresponsetodeterminingthattheexternalresourceisalocally-installedapplication106,themessagingclient104instructstheclientdevice102tolaunchtheexternalresourcebyexecutinglocally-storedcodecorrespondingtotheexternalresource.Inresponsetodeterminingthattheexternalresourceisaweb-basedresource,themessagingclient104communicateswiththethird-partyservers110(forexample)toobtainamarkup-languagedocumentcorrespondingtotheselectedexternalresource.Themessagingclient104thenprocessestheobtainedmarkup-languagedocumenttopresenttheweb-basedexternalresourcewithinauserinterfaceofthemessagingclient104.
Themessagingclient104canpresentalistoftheavailableexternalresources(e.g.,applications106orapplets)toausertolaunchoraccessagivenexternalresource.Thislistcanbepresentedinacontext-sensitivemenu.Forexample,theiconsrepresentingdifferentonesoftheapplication106(orapplets)canvarybasedonhowthemenuislaunchedbytheuser(e.g.,fromaconversationinterfaceorfromanon-conversationinterface).
FIG.2isablockdiagramillustratingfurtherdetailsregardingthemessagingsystem100,accordingtosomeexamples.Specifically,themessagingsystem100isshowntocomprisethemessagingclient104andtheapplicationservers114.Themessagingsystem100embodiesanumberofsubsystems,whicharesupportedontheclient-sidebythemessagingclient104andontheserver-sidebytheapplicationservers114.Thesesubsystemsinclude,forexample,anephemeraltimersystem202,acollectionmanagementsystem204,anaugmentationsystem208,amapsystem210,anexternalresourcesystem212,andavisualcodescannersystem214.
Theephemeraltimersystem202isresponsibleforenforcingthetemporaryortime-limitedaccesstocontentbythemessagingclient104andthemessagingserver118.Theephemeraltimersystem202incorporatesanumberoftimersthat,basedondurationanddisplayparametersassociatedwithamessage,orcollectionofmessages(e.g.,astory),selectivelyenableaccess(e.g.,forpresentationanddisplay)tomessagesandassociatedcontentviathemessagingclient104.Furtherdetailsregardingtheoperationoftheephemeraltimersystem202areprovidedbelow.
Thecollectionmanagementsystem204isresponsibleformanagingsetsorcollectionsofmedia(e.g.,collectionsoftext,imagevideo,andaudiodata).Acollectionofcontent(e.g.,messages,includingimages,video,text,andaudio)maybeorganizedintoan“eventgallery”oran“eventstory.”Suchacollectionmaybemadeavailableforaspecifiedtimeperiod,suchasthedurationofaneventtowhichthecontentrelates.Forexample,contentrelatingtoamusicconcertmaybemadeavailableasa“story”forthedurationofthatmusicconcert.Thecollectionmanagementsystem204mayalsoberesponsibleforpublishinganiconthatprovidesnotificationoftheexistenceofaparticularcollectiontotheuserinterfaceofthemessagingclient104.
Thecollectionmanagementsystem204furthermoreincludesacurationinterface206thatallowsacollectionmanagertomanageandcurateaparticularcollectionofcontent.Forexample,thecurationinterface206enablesaneventorganizertocurateacollectionofcontentrelatingtoaspecificevent(e.g.,deleteinappropriatecontentorredundantmessages).Additionally,thecollectionmanagementsystem204employsmachinevision(orimagerecognitiontechnology)andcontentrulestoautomaticallycurateacontentcollection.Incertainexamples,compensationmaybepaidtoauserfortheinclusionofuser-generatedcontentintoacollection.Insuchcases,thecollectionmanagementsystem204operatestoautomaticallymakepaymentstosuchusersfortheuseoftheircontent.
Insomeexamples,theaugmentationsystem208providesauser-basedpublicationplatformthatenablesuserstoselectageolocationonamapanduploadcontentassociatedwiththeselectedgeolocation.Theusermayalsospecifycircumstancesunderwhichaparticularmediaoverlayshouldbeofferedtootherusers.Theaugmentationsystem208generatesamediaoverlaythatincludestheuploadedcontentandassociatestheuploadedcontentwiththeselectedgeolocation.
Inotherexamples,theaugmentationsystem208providesamerchant-basedpublicationplatformthatenablesmerchantstoselectaparticularmediaoverlayassociatedwithageolocationviaabiddingprocess.Forexample,theaugmentationsystem208associatesthemediaoverlayofthehighestbiddingmerchantwithacorrespondinggeolocationforapredefinedamountoftime.
Inotherexamples,asdiscussedbelowwithrespecttoFIG.3,theaugmentationsystem208providesforpresentingaugmentedrealitycontentinassociationwithanimageoravideocapturedbyacameraoftheclientdevice102.Theaugmentationsystem208mayimplementorotherwiseaccessaugmentedrealitycontentitems(e.g.,correspondingtoapplyingLensesoraugmentedrealityexperiences)forprovidingreal-timespecialeffect(s)and/orsound(s)thatmaybeaddedtotheimageorvideo.
Themapsystem210providesvariousgeographiclocationfunctions,andsupportsthepresentationofmap-basedmediacontentandmessagesbythemessagingclient104.Forexample,themapsystem210enablesthedisplayofusericonsoravatars(e.g.,storedinprofiledata302)onamaptoindicateacurrentorpastlocationof“friends”ofauser,aswellasmediacontent(e.g.,collectionsofmessagesincludingphotographsandvideos)generatedbysuchfriends,withinthecontextofamap.Forexample,amessagepostedbyausertothemessagingsystem100fromaspecificgeographiclocationmaybedisplayedwithinthecontextofamapatthatparticularlocationto“friends”ofaspecificuseronamapinterfaceofthemessagingclient104.Ausercanfurthermoresharehisorherlocationandstatusinformation(e.g.,usinganappropriatestatusavatar)withotherusersofthemessagingsystem100viathemessagingclient104,withthislocationandstatusinformationbeingsimilarlydisplayedwithinthecontextofamapinterfaceofthemessagingclient104toselectedusers.
InordertointegratethefunctionsoftheSDKintotheweb-basedresource,theSDKisdownloadedbyathird-partyserver110fromthemessagingserver118orisotherwisereceivedbythethird-partyserver110.Oncedownloadedorreceived,theSDKisincludedaspartoftheapplicationcodeofaweb-basedexternalresource.Thecodeoftheweb-basedresourcecanthencallorinvokecertainfunctionsoftheSDKtointegratefeaturesofthemessagingclient104intotheweb-basedresource.
TheSDKstoredonthemessagingserver118effectivelyprovidesthebridgebetweenanexternalresource(e.g.,applications106orappletsandthemessagingclient104.Thisprovidestheuserwithaseamlessexperienceofcommunicatingwithotherusersonthemessagingclient104,whilealsopreservingthelookandfeelofthemessagingclient104.Tobridgecommunicationsbetweenanexternalresourceandamessagingclient104,incertainexamples,theSDKfacilitatescommunicationbetweenthird-partyservers110andthemessagingclient104.Incertainexamples,aWebViewJavaScriptBridgerunningonaclientdevice102establishestwoone-waycommunicationchannelsbetweenanexternalresourceandthemessagingclient104.Messagesaresentbetweentheexternalresourceandthemessagingclient104viathesecommunicationchannelsasynchronously.EachSDKfunctioninvocationissentasamessageandcallback.EachSDKfunctionisimplementedbyconstructingauniquecallbackidentifierandsendingamessagewiththatcallbackidentifier.
ByusingtheSDK,notallinformationfromthemessagingclient104issharedwiththird-partyservers110.TheSDKlimitswhichinformationissharedbasedontheneedsoftheexternalresource.Incertainexamples,eachthird-partyserver110providesanHTML5filecorrespondingtotheweb-basedexternalresourcetothemessagingserver118.Themessagingserver118canaddavisualrepresentation(suchasaboxartorothergraphic)oftheweb-basedexternalresourceinthemessagingclient104.Oncetheuserselectsthevisualrepresentationorinstructsthemessagingclient104throughaGUIofthemessagingclient104toaccessfeaturesoftheweb-basedexternalresource,themessagingclient104obtainstheHTML5fileandinstantiatestheresourcesnecessarytoaccessthefeaturesoftheweb-basedexternalresource.
Themessagingclient104controlsthetypeofuserdatathatissharedwithexternalresourcesbasedonthetypeofexternalresourcebeingauthorized.Forexample,externalresourcesthatincludefull-scaleapplications(e.g.,anapplication106)areprovidedwithaccesstoafirsttypeofuserdata(e.g.,onlytwo-dimensionalavatarsofuserswithorwithoutdifferentavatarcharacteristics).Asanotherexample,externalresourcesthatincludesmall-scaleversionsofapplications(e.g.,web-basedversionsofapplications)areprovidedwithaccesstoasecondtypeofuserdata(e.g.,paymentinformation,two-dimensionalavatarsofusers,three-dimensionalavatarsofusers,andavatarswithvariousavatarcharacteristics).Avatarcharacteristicsincludedifferentwaystocustomizealookandfeelofanavatar,suchasdifferentposes,facialfeatures,clothing,andsoforth.
Thevisualcodescannersystem214implementsvariousfunctionsforscanningvisualcodeswithrespecttothemessagingsystem100.Forexample,visualcodesmaycorrespondtoQRcode,1Barcode,DataMatrix,Azteccode,andthelike.Inoneormoreembodiments,theclientdevice102mayaccessimagedata(e.g.,astillimage,avideo)thatincludesthevisualcode.Forexample,theimagedatamaybecapturedbytheclientdevice102inrealtime(e.g.,aliveimagefeedfromacamerasensoroftheclientdevice102).Inanotherexample,theimagedataisstoredonorreceivedbytheclientdevice102,basedonpriorimagecaptureviatheclientdevice102oranotherdevice.Thevisualcodescannersystem214isconfiguredtoaccesstheimagedatathatincludesthevisualcode.Inembodimentswheretheimagedataisvideo,thevisualcodescannersystem214isconfiguredtoanalyzeindividualframesofthevideooracombinationofmultipleframesofthevideotodetectanddecodethevisualcode.
Subsequenttoidentifyingthecustomgraphic,thevisualcodescannersystem214usesthecustomgraphicasanalignmentpatternfordecoding.Forinstance,thevisualcodescannersystem214extractsspatialattributesofthecustomgraphicintheimageandcomparestheextractedspatialattributestoreferencespatialattributestodetermineanalignmentofthecustomgraphic.Thevisualcodescannersystem214maythengenerateatransformedimageoftheimageaccordingtothealignment(e.g.,arotationorde-skew).Aftergeneratingthetransformedimage,thevisualcodescannersystem214decodesthedataencodedinaportionofthetransformedimage.Forexample,thedotsofthevisualcodearetransformedintodatashownasonesfordotsandzerosfornon-dots,althoughthisismerelyanillustrativeexampleandotherschemescanbeemployed.Inthisway,visualcodescannersystem214usesthecustomgraphicincludedinthevisualcodeasoneormorefunctionalpatternssuchasafinderpatternoranalignmentpattern.
Basedondecodingthedataencodedintheimage,thevisualcodescannersystem214facilitatesaccesstofeaturesand/oruniquecontentofthemessagingsystem100.Thedataencodedinaportionoftheimagecanindicateaparticularfeature(e.g.,actiontoperform)orincludeinformationtobeusedinconjunctionwithaparticularfeature.Examplefeaturesinclude,butarenotlimitedto:accessing/activatinganaugmentedrealitycontentitem(e.g.,unlockingaLens),addingfriends,linkingtoawebsite,connectingtoparticularcontent,andthelike.Forexample,auniquevisualcodemaybeassignedtoeachaugmentedrealitycontentitem(e.g.,upongenerationoftheaugmentedrealitycontentitem),friend,websiteand/orothercontent,toprovideaccesstheretobydecodingthecorrespondinguniquevisualcode.
FIG.3isaschematicdiagramillustratingdatastructures300,whichmaybestoredinthedatabase126ofthemessagingserversystem108,accordingtocertainexamples.Whilethecontentofthedatabase126isshowntocompriseanumberoftables,itwillbeappreciatedthatthedatacouldbestoredinothertypesofdatastructures(e.g.,asanobject-orienteddatabase).
Anentitytable308storesentitydata,andislinked(e.g.,referentially)toanentitygraph304andprofiledata302.Entitiesforwhichrecordsaremaintainedwithintheentitytable308mayincludeindividuals,corporateentities,organizations,objects,places,events,andsoforth.Regardlessofentitytype,anyentityregardingwhichthemessagingserversystem108storesdatamaybearecognizedentity.Eachentityisprovidedwithauniqueidentifier,aswellasanentitytypeidentifier(notshown).
Theentitygraph304storesinformationregardingrelationshipsandassociationsbetweenentities.Suchrelationshipsmaybesocial,professional(e.g.,workatacommoncorporationororganization)interested-basedoractivity-based,merelyforexample.
Theprofiledata302storesmultipletypesofprofiledataaboutaparticularentity.Theprofiledata302maybeselectivelyusedandpresentedtootherusersofthemessagingsystem100,basedonprivacysettingsspecifiedbyaparticularentity.Wheretheentityisanindividual,theprofiledata302includes,forexample,ausername,telephonenumber,address,settings(e.g.,notificationandprivacysettings),aswellasauser-selectedavatarrepresentation(orcollectionofsuchavatarrepresentations).Aparticularusermaythenselectivelyincludeoneormoreoftheseavatarrepresentationswithinthecontentofmessagescommunicatedviathemessagingsystem100,andonmapinterfacesdisplayedbymessagingclients104tootherusers.Thecollectionofavatarrepresentationsmayinclude“statusavatars,”whichpresentagraphicalrepresentationofastatusoractivitythattheusermayselecttocommunicateataparticulartime.
Wheretheentityisagroup,theprofiledata302forthegroupmaysimilarlyincludeoneormoreavatarrepresentationsassociatedwiththegroup,inadditiontothegroupname,members,andvarioussettings(e.g.,notifications)fortherelevantgroup.
Inoneormoreembodiments,theprofiledata302includesarespectivevisualcodeforeachentity.Forexample,thevisualcodeforaparticularentityisscannableinordertoaccess(e.g.,add)thatentityasafriend.
Thedatabase126alsostoresaugmentationdata,suchasoverlaysorfilters,inanaugmentationtable310.Theaugmentationdataisassociatedwithandappliedtovideos(forwhichdataisstoredinavideotable314)andimages(forwhichdataisstoredinanimagetable316).
Filters,inoneexample,areoverlaysthataredisplayedasoverlaidonanimageorvideoduringpresentationtoarecipientuser.Filtersmaybeofvarioustypes,includinguser-selectedfiltersfromasetoffilterspresentedtoasendinguserbythemessagingclient104whenthesendinguseriscomposingamessage.Othertypesoffiltersincludegeolocationfilters(alsoknownasgeo-filters),whichmaybepresentedtoasendinguserbasedongeographiclocation.Forexample,geolocationfiltersspecifictoaneighborhoodorspeciallocationmaybepresentedwithinauserinterfacebythemessagingclient104,basedongeolocationinformationdeterminedbyaGlobalPositioningSystem(GPS)unitoftheclientdevice102.
Anothertypeoffilterisadatafilter,whichmaybeselectivelypresentedtoasendinguserbythemessagingclient104,basedonotherinputsorinformationgatheredbytheclientdevice102duringthemessagecreationprocess.Examplesofdatafiltersincludecurrenttemperatureataspecificlocation,acurrentspeedatwhichasendinguseristraveling,batterylifeforaclientdevice102,orthecurrenttime.
Otheraugmentationdatathatmaybestoredwithintheaugmentationtable310(e.g.,inconjunctionwiththeimagetable316and/orthevideotable314)includesaugmentedrealitycontentitems(e.g.,correspondingtoapplyingLensesoraugmentedrealityexperiences).Anaugmentedrealitycontentitemmayprovideareal-timespecialeffectand/orsoundthatmaybeaddedtoanimageoravideo.Eachoftheaugmentedrealitycontentitemsmaybeavailableforselectionandactivationbythemessagingclient104.
Real-timevideoprocessingcanbeperformedwithanykindofvideodata(e.g.,videostreams,videofiles,etc.)savedinamemoryofacomputerizedsystemofanykind.Forexample,ausercanloadvideofilesandsavetheminamemoryofadevice,orcangenerateavideostreamusingsensorsofthedevice.Additionally,anyobjectscanbeprocessedusingacomputeranimationmodel,suchasahuman'sfaceandpartsofahumanbody,animals,ornon-livingthingssuchaschairs,cars,orotherobjects.
Insomeexamplesofacomputeranimationmodeltotransformimagedatausingfacedetection,thefaceisdetectedonanimagewithuseofaspecificfacedetectionalgorithm(e.g.,Viola-Jones).Then,anActiveShapeModel(ASM)algorithmisappliedtothefaceregionofanimagetodetectfacialfeaturereferencepoints.
Othermethodsandalgorithmssuitableforfacedetectioncanbeused.Forexample,insomeexamples,featuresarelocatedusingalandmark,whichrepresentsadistinguishablepointpresentinmostoftheimagesunderconsideration.Forfaciallandmarks,forexample,thelocationofthelefteyepupilmaybeused.Ifaninitiallandmarkisnotidentifiable(e.g.,ifapersonhasaneyepatch),secondarylandmarksmaybeused.Suchlandmarkidentificationproceduresmaybeusedforanysuchobjects.Insomeexamples,asetoflandmarksformsashape.Shapescanberepresentedasvectorsusingthecoordinatesofthepointsintheshape.Oneshapeisalignedtoanotherwithasimilaritytransform(allowingtranslation,scaling,androtation)thatminimizestheaverageEuclideandistancebetweenshapepoints.Themeanshapeisthemeanofthealignedtrainingshapes.
Insomeexamples,asearchforlandmarksfromthemeanshapealignedtothepositionandsizeofthefacedeterminedbyaglobalfacedetectorisstarted.Suchasearchthenrepeatsthestepsofsuggestingatentativeshapebyadjustingthelocationsofshapepointsbytemplatematchingoftheimagetexturearoundeachpointandthenconformingthetentativeshapetoaglobalshapemodeluntilconvergenceoccurs.Insomesystems,individualtemplatematchesareunreliable,andtheshapemodelpoolstheresultsoftheweaktemplatematchestoformastrongeroverallclassifier.Theentiresearchisrepeatedateachlevelinanimagepyramid,fromcoarsetofineresolution.
Atransformationsystemcancaptureanimageorvideostreamonaclientdevice(e.g.,theclientdevice102)andperformcompleximagemanipulationslocallyontheclientdevice102whilemaintainingasuitableuserexperience,computationtime,andpowerconsumption.Thecompleximagemanipulationsmayincludesizeandshapechanges,emotiontransfers(e.g.,changingafacefromafrowntoasmile),statetransfers(e.g.,agingasubject,reducingapparentage,changinggender),styletransfers,graphicalelementapplication,andanyothersuitableimageorvideomanipulationimplementedbyaconvolutionalneuralnetworkthathasbeenconfiguredtoexecuteefficientlyontheclientdevice102.
Thegraphicaluserinterface,presentingthemodificationperformedbythetransformsystem,maysupplytheuserwithadditionalinteractionoptions.Suchoptionsmaybebasedontheinterfaceusedtoinitiatethecontentcaptureandselectionofaparticularcomputeranimationmodel(e.g.,initiationfromacontentcreatoruserinterface).Invariousexamples,amodificationmaybepersistentafteraninitialselectionofamodificationicon.Theusermaytogglethemodificationonoroffbytappingorotherwiseselectingthefacebeingmodifiedbythetransformationsystemandstoreitforlaterviewingorbrowsetootherareasoftheimagingapplication.Wheremultiplefacesaremodifiedbythetransformationsystem,theusermaytogglethemodificationonoroffgloballybytappingorselectingasinglefacemodifiedanddisplayedwithinagraphicaluserinterface.Insomeexamples,individualfaces,amongagroupofmultiplefaces,maybeindividuallymodified,orsuchmodificationsmaybeindividuallytoggledbytappingorselectingtheindividualfaceoraseriesofindividualfacesdisplayedwithinthegraphicaluserinterface.
Inoneormoreembodiments,theaugmentationtable310storesarespectivevisualcodeforeachaugmentedrealitycontentitem(oroverlay,imagetransformation,ARimage,andthelike)thatmaybeappliedtoimagedata.Forexample,thevisualcodeforaparticularaugmentedrealitycontentitemisscannableinordertoaccess(e.g.,activate,unlock)thataugmentedrealitycontentitem.
Astorytable312storesdataregardingcollectionsofmessagesandassociatedimage,video,oraudiodata,whicharecompiledintoacollection(e.g.,astoryoragallery).Thecreationofaparticularcollectionmaybeinitiatedbyaparticularuser(e.g.,eachuserforwhicharecordismaintainedintheentitytable308).Ausermaycreatea“personalstory”intheformofacollectionofcontentthathasbeencreatedandsent/broadcastbythatuser.Tothisend,theuserinterfaceofthemessagingclient104mayincludeaniconthatisuser-selectabletoenableasendingusertoaddspecificcontenttohisorherpersonalstory.
Acollectionmayalsoconstitutea“livestory,”whichisacollectionofcontentfrommultipleusersthatiscreatedmanually,automatically,orusingacombinationofmanualandautomatictechniques.Forexample,a“livestory”mayconstituteacuratedstreamofuser-submittedcontentfromvarieslocationsandevents.Userswhoseclientdeviceshavelocationservicesenabledandareatacommonlocationeventataparticulartimemay,forexample,bepresentedwithanoption,viaauserinterfaceofthemessagingclient104,tocontributecontenttoaparticularlivestory.Thelivestorymaybeidentifiedtotheuserbythemessagingclient104,basedonhisorherlocation.Theendresultisa“livestory”toldfromacommunityperspective.
Afurthertypeofcontentcollectionisknownasa“locationstory,”whichenablesauserwhoseclientdevice102islocatedwithinaspecificgeographiclocation(e.g.,onacollegeoruniversitycampus)tocontributetoaparticularcollection.Insomeexamples,acontributiontoalocationstorymayrequireaseconddegreeofauthenticationtoverifythattheenduserbelongstoaspecificorganizationorotherentity(e.g.,isastudentontheuniversitycampus).
Asmentionedabove,thevideotable314storesvideodatathat,inoneexample,isassociatedwithmessagesforwhichrecordsaremaintainedwithinthemessagetable306.Similarly,theimagetable316storesimagedataassociatedwithmessagesforwhichmessagedataisstoredintheentitytable308.Theentitytable308mayassociatevariousaugmentationsfromtheaugmentationtable310withvariousimagesandvideosstoredintheimagetable316andthevideotable314.
FIG.4isaschematicdiagramillustratingastructureofamessage400,accordingtosomeexamples,generatedbyamessagingclient104forcommunicationtoafurthermessagingclient104orthemessagingserver118.Thecontentofaparticularmessage400isusedtopopulatethemessagetable306storedwithinthedatabase126,accessiblebythemessagingserver118.Similarly,thecontentofamessage400isstoredinmemoryas“in-transit”or“in-flight”dataoftheclientdevice102ortheapplicationservers114.Amessage400isshowntoincludethefollowingexamplecomponents:
Inoneormoreembodiments,themessagingclient104isrunningonaclientdevice102associatedwithauserofmessagingserversystem108.Theuserisassociatedwithauseraccountofthemessagingserversystem108.Forexample,theuserisidentifiedbythemessagingserversystem108basedononeormoreuniqueidentifiers(e.g.,amessagingsystemidentifier,emailaddressand/oradeviceidentifier)associatedwiththeuseraccountsfortheuser.Insomeembodiments,themessagingserversystem108implementsand/orworksinconjunctionwithasocialnetworkserver124whichisconfiguredtoidentifyotherusers(e.g.,friends)withwhichtheuserofthemessagingclient104hasrelationships.
Asnotedabove,eachofthevisualcodescannersystem214andtheaugmentationsystem208maycorrespondtorespectivesubsystemsofthemessagingsystem100,andmaybesupportedontheclientsidebythemessagingclient104and/orontheserversidebytheapplicationservers114.Thus,theoperationsdescribedhereinwithrespecttothevisualcodescannersystem214andtheaugmentationsystem208maybeimplementedclientside,serversideand/oracombinationofclientsideandserverside.
Asdescribedherein,themessagingsystem100providesforausertointeractwithvisualcodes,toaccessfeaturesofthemessagingsystem100.Inresponsetouserselectionofavisualcodeinacapturedimage,themessagingclient104providesforupdatingacustomgraphicwithinthevisualcode.Themessagingclient104animatesmovementofthecustomgraphicfromthevisualcodetoacarouselinterfacewithuser-selectableiconsforrespectiveaugmentedrealitycontent,andfurtherupdatesthecarouselinterfacewithaniconcorrespondingtothevisualcode.Thecode(e.g.,assets)foreffectingthecorrespondingaugmentedrealitycontentitemcanbeautomaticallydownloadedand/oractivated,therebyreducingtheamountofinteractionrequiredfromtheuser.
Atblock502,themessagingclient104runningonthemessagingclient104displayscapturedimagedatathatincludesavisualcodewithacustomgraphicandmarkings.Asnotedabove,themessagingclient104isconfiguredaccessimagedata(e.g.,astillimage,avideo)thatincludesthevisualcode.Theimagedatamaybecapturedbytheclientdevice102inrealtime(e.g.,aliveimagefeedfromacamerasensoroftheclientdevice102).Forexample,themessagingclient104mayactivateacamerauponinitiationofthemessagingclient104.Themessagingclient104mayalsoincludeinterfaceelementsforactivatingthecamera,andforswitchingbetweenafront-facingandrear-facingcamera.
Asanalternativetoreal-timecaptureofimagedata,theimagedatamaycorrespondtoapriorimagecapturedbytheclientdevice102orbyanotherdevice.Forexample,thepriorimagemayhavebeensentasamessagetotheuserofthemessagingclient104,ortheusermayhaveselectedtoview/downloadthepriorimageviathemessagingclient104.
Thecapturedimagedatamaycorrespondtoanobject,suchasaposter,acomputerdisplay,anarticleofclothing,orapieceofprintedpaper.Thevisualcodemaybeprintedorotherwiseappearontheobject.Inoneormoreembodiments,thevisualcodeissquare-shapedwithroundededges.
Inoneormoreembodiments,thevisualcodeincludesacustomgraphicandmarking.Thecustomgraphiccanbeemployedasafinderpatternusedforidentificationandrecognitionofavisualcode,oranalignmentpatternusedtofacilitatedecoding.InthebelowexamplesofFIGS.6-10,thecustomgraphicisdepictedasaghosticon.However,otherexamplecustomgraphicsmaybeused,andcanincludedesignswithmultiplepaths,multiplepolygons,multipleaestheticelements,orotherdesignfeatures.
Thus,themessagingclient104,inconjunctionwiththevisualcodescannersystem214,identifiesthecustomgraphic(e.g.,aghosticon)intheimagedataandusesthecustomgraphicasanalignmentpatternfordecodingdataincludedinthevisualcode.Asnotedabove,thevisualcodescannersystem214extractscandidateshapefeaturesfromtheimagedata,appliesrulesandcriteriatofilteroutirrelevantshapefeatures,andperformscomparisonswithareferenceshapefeaturebasedonamatchscorethatexceedsathreshold.Thevisualcodescannersystem214furtherusesthecustomgraphicasanalignmentpatternfordecoding,andgeneratesatransformedimageoftheimagedataaccordingtothealignment.
Asnotedabove,thevisualcodefurtherincludesmarkings(e.g.,dots)arrangedinauniqueconfigurationwhichisdecodabletoaccessaparticularfeatureofthemessagingsystem100.IntheexampleofFIG.5,thefeaturecorrespondstoanaugmentedrealitycontentitem.However,thefeaturesasdescribedhereinareisnotlimitedtoaugmentedrealitycontentitems,andmayinsteadrelatetootherfeaturessuchasaddingafriend,linkingtoawebsite,connectingtoparticularcontent,andthelike.
Thus,atblock502,themessagingclient104displaysavisualcodeincludedwithincapturedimagedata.Thevisualcodeincludesacustomgraphic(e.g.,whichmaybetransformedfordisplaybasedonalignment)andmarkings.Atblock504,theuseroftheclientdevice102selects(e.g.,viaataporpressgesture)thevisualcodeasdisplayedonthemessagingclient104.Inresponsetosuchuserselection,thevisualcodescannersystem214decodesthemarkingsofthevisualcode.Bywayofnon-limitingexample,themarkings(e.g.,dots)ofthevisualcodearedecodedintodatashownasonesfordotsandzerosfornon-dots,toidentifyaparticularfeature(e.g.,augmentedrealitycontentitem).
Atblock506,themessagingclient104displaysanupdatedcustomgraphic.Forexample,themessagingclient104causesthecustomgraphictobemodifiedtoindicatetheuserselectionreceivedatblock504.Bywayofnon-limitingexample,andasdiscussedbelowwithrespecttoFIGS.7-8,theoriginalcustomgraphicofaghosticonwithtwoopeneyesisupdatedtowink.Updatingthecustomgraphicmaycorrespondwithreplacingtheoriginalcustomgraphic(e.g.,thenon-winkingghosticon)withastaticimage(e.g.,thewinkingghosticon).Alternatively,updatingthecustomgraphicmaycorrespondwithanimatingthecustomgraphic(e.g.,animatingtheghosticontowinkortoperformanotherpredefinedaction).
Inoneexample,theupdatedcustomgraphicisadefaultgraphic(e.g.,adefaultstaticimageoradefaultanimation)thatispredefined(e.g.,byasystemadministrator).Inanotherexample,theupdatedcustomgraphicmaybesetbasedonthecode(e.g.,assets)forthecorrespondingaugmentedrealitycontentitem.Asdescribedherein,thevisualcodecorrespondswithanaugmentedrealitycontentitemthathascodeassociatedtherewith.Thecodeincludesthestoredassetsforimplementingtheaugmentedreality(e.g.,visualand/oraudioeffects)correspondingtotheaugmentedrealitycontentitem.Inaddition,thecodemayincludeaspecificiconfortheaugmentedrealitycontentitem,wherethespecificiconisusablewithinacarouselinterface.
Inthisregard,asdiscussedfurtherbelowwithrespecttoFIG.9,themessagingclient104atblock508providesforanimatingthecustomgraphicasmovingfromwithinthedisplayareaofthevisualcodetoacarouselinterface.Inoneormoreembodiments,thecarouselinterfaceprovidedbythemessagingclient104allowstheusertocyclethroughand/orselectadifferentaugmentedrealitycontentitem(e.g.,Lens)toapplywithrespecttocapturedimagedata.Eachoftheavailableaugmentedrealitycontentitemsisrepresentedbyaspecificicon(e.g.,acircularicon)whichisuser-selectableforswitchingtotherespectiveaugmentedrealitycontentitem.Asnotedabove,eachoftheaugmentedrealitycontentitemshascorrespondingcodewhichimplementstheeffect(e.g.,videoand/oraudioeffect)andthespecificiconforthataugmentedrealitycontentitem.
Aniconcorrespondingtoacurrentlyactiveaugmentedrealitycontentitemmaybedisplayedinadifferentmannerrelativetotheremainingicons.Forexample,theiconfortheactiveaugmentedrealitycontentitemisdisplayedascentered,largerthanand/orhighlightedrelativetotheremainingiconsinthecarouselinterface.Inoneormoreembodiments,theremainingiconsaredisplayedinarankedorder,forexample,basedonpredefinedrankings(e.g.,oneormoreofpopularity,personalhistory,promotions,andthelike).
Inaddition,themessagingclient104atblock510updatesthecarouselinterfacetoincludeanadditionaliconfortheaugmentedrealitycontentitemcorrespondingtothevisualcode.Forexample,themessagingclient104providesananimationwhichdepictstheadditionaliconasbeinginsertedbetweentwoexistingiconsofthecarouselinterface.
Inoneormoreembodiments,themessagingclient104mayprovideforupdatingthecustomgraphic(e.g.,block506),movingthevisualcodetowardthecarouselinterface(e.g.,block508),andinsertingtheadditionaliconintothecarouselinterface(e.g.,block510)tobeasmooth,animatedtransition.Forexample,themessagingclient104causesthecustomgraphictomovetowardsthecarouselinterface,andtotransform(e.g.,scale,rotate,andthelike)whilemoving.Inaddition,themessagingclient104providesforanimatingtheexistingiconsinthecarouselinterfacetomakespacefortheadditionalicon,andforsmoothlychangingthecustomgraphicfromtheonethatisvisibleonthecode(e.g.,theoriginalghosticon)tothespecificiconcreatedfortheaugmentedrealitycontentitem.
Atoperation512,themessagingclient104sends,tothemessagingserversystem108,arequestforthecodeoftheaugmentedrealitycontentitemcorrespondingtothevisualcode.Inresponse,themessagingserversystem108sends,tothemessagingclient104,thecodeoftheaugmentedrealitycontentitemcorrespondingtothevisualcode(operation514).
Asnotedabove,thecodemayincludetheassetsforimplementing/effectingtheaugmentedrealitycontentitem.Inoneormoreembodiments,thecodeisstoredintheaugmentationtable310ofthedatabase126.Therequestofoperation512mayincludeanidentifierfortheaugmentedrealitycontentitem,wheretheidentifierisbasedonthedecodingofthemarkingsasdescribedabove.
Inresponsetoreceivingthecode,themessagingclient104providesforactivatingtheaugmentedrealitycontentitemwithrespecttothecapturedimagedata(block516).Asnotedabove,activatingtheaugmentedrealitycontentitemincludespresentingthevisualand/oraudioeffectsassociatedwiththeaugmentedrealitycontentiteminconjunctionwiththecapturedimagedata.
AsdiscussedfurtherbelowwithrespecttoFIGS.10A-10C,themessagingclient104mayimplementdifferentoptionsfordownloadingthecodefortheaugmentedrealitycontentitem,and/orforactivatingtheaugmentedrealitycontentitem.Selectionoftheoptionmaybedeterminedbytheuserorasystemadministrator,basedonappropriateinterfacesprovidedbythemessagingsystem100.
Inafirstoption,themessagingclient104requeststhecode(e.g.,peroperation512),anddownloadsthecode(e.g.,peroperation514)correspondingtotheaugmentedrealitycontentiteminresponsetotheuserinputselectingthevisualcodeatblock504.Inaddition,themessagingclient104activatestheaugmentedrealitycontentitem(e.g.,perblock516)inresponsetoseconduserinput(notshowninFIG.5)thatselectstheadditionaliconthatwasinsertedinthecarouselinterface(e.g.,perblock510).
Inasecondoption,themessagingclient104requeststhecode(e.g.,peroperation512),downloadsthecode(e.g.,peroperation514)correspondingtotheaugmentedrealitycontentitem,andactivatestheaugmentedrealitycontentitem(e.g.,perblock516),allinresponsetotheuserinputselectingthecodeatblock504.Thus,thesecondoptiondiffersfromthefirstoptioninthatthesecondoptiondoesnotrequiretheseconduserinputtoactivatetheaugmentedrealitycontentitem.
Inathirdoption,themessagingclient104requeststhecode(e.g.,peroperation512),anddownloadsthecode(e.g.,peroperation514)correspondingtotheaugmentedrealitycontentiteminresponsetoseconduserinput(notshowninFIG.5)thatselectstheadditionaliconinsertedinthecarouselinterface(e.g.,perblock510).Forexample,theadditionaliconisinitiallygrayedoutwheninsertedintothecarouselinterface.Userselectionofthegrayedouticoncausesthemessagingclient104torequestanddownloadthecodecorrespondingtotheaugmentedrealitycontentitem.Themessagingclient104mayprovideananimationtoshowdownloadprogress.Forexample,theanimationmayprogressivelyremovethegraymaskfortheicontoindicatedownloadprogressandcompletion.
WhiletheexampleofFIG.5relatestointeractionwithrespecttovisualcodesforaugmentedrealitycontentitems,themessagingsystem100isnotlimitedtosuch.Themessagingsystem100mayprovideforsimilarinteractionwithrespecttovisualcodesforotherfeaturesprovidebythemessagingsystem100.Examplesofothersuchfeaturesinclude,butarenotlimitedto,addingafriend,linkingtoawebsite,connectingtoparticularcontent,andthelike.
Thus,themessagingsystem100asdescribedhereinprovidesforinteractingwithvisualcodesforaccessingfeaturesofthemessagingsystem100.Inothersystems,afterselectingavisualcode,theuserispresentedwithaseparatemessagebox(e.g.,overlay)thatpromptstheusertodownloadand/oractivatethefeature.Thispromptmaybedistracting,asitswitchestheuser'sattentionawayfromthecurrentinterfacerelatedtothefeature.Themessagingsystem100providesforamoreseamlesstransitionindownloadingand/oractivatingthefeature.Withoutprovidingforthevisualcodeinteractionsasdescribedherein,endusersmaybeprovidedwithadditionalpromptsthatcanbedistractingandinterferewithuserinterfaceflow.Assuch,themessagingsystem100savestimefortheuser,and/orreducescomputationalresources/processingpowerassociatedwithadditionalpromptsandassociateddelays.Moreover,thevisualcodeinteractionsasdescribedhereinmaybemoreengagingforusersofthemessagingsystem100.
FIG.6illustratesauserinterface602inwhichavisualcodeisdisplayedwithincapturedimagedata,inaccordancewithsomeexampleembodiments.AsshownintheexampleofFIG.6,theuserinterface602includesavisualcode604,markings606,acustomgraphic608,capturedimagedata610andacarouselinterface612.
Inoneormoreembodiments,theuserinterface602isprovidedbythemessagingclient104.Theuserinterface602providesfordisplayingaugmentedrealitycontentinassociationwiththecapturedimagedata610.Asnotedabove,thecapturedimagedata610maycorrespondtoimagedatacapturedbyafront-facingcameraorarear-facingcameraoftheclientdevice102.Alternatively,theimagedatamaycorrespondtoapriorimagecapturedviatheclientdevice102oranotherdevice.
Theaugmentedrealitycontentdisplayedinassociationwiththecapturedimagedata610isbasedonacurrently-selectedaugmentedrealitycontentitem(e.g.,Lens).Inthisregard,thecarouselinterface612includesasetoficons,eachofwhichisuser-selectable(e.g.,byatap/pressgesture)toselect/activatearespectiveaugmentedrealitycontentitem.Thecarouselinterface612allowstheusertocyclethroughtheicons(e.g.,viaswipegestures),andtoselectaparticularaugmentedrealitycontentitemtoapplywithrespecttothecapturedimagedata610.
WhilenotshownintheexampleofFIG.6,eachiconmayhavearespectivegraphictoindicateitsrespectiveaugmentedrealitycontentitem(e.g.,ahairgraphictoindicateahaireffect,asmilinggraphictoindicateasmileeffect,andthelike).Theiconfortheactiveaugmentedrealitycontentitemmaybecentered,largerthanand/orhighlightedrelativetotheremainingiconsinthecarouselinterface612.
IntheexampleofFIG.6,thecapturedimagedata610includesavisualcode604.Forexample,thecapturedimagedata610maycorrespondtoanobject(e.g.,aposter,acomputerdisplay,anarticleofclothing,orapieceofprintedpaper),andthevisualcode604mayappearontheobject.Thevisualcodeincludesthecustomgraphic608andmarkings606.
Asnotedabove,thecustomgraphic608maybeemployedasafinderpatternusedforidentificationandrecognitionofthevisualcode604.IntheexampleofFIG.6,thecustomgraphic608isdepictedasaghosticon.However,othercustomgraphicsmaybeused.
Themarkings606ofthevisualcode604arearrangedinauniqueconfigurationwhichisdecodabletoaccessaparticularfeatureofthemessagingsystem100.IntheexampleofFIG.6,thefeaturecorrespondstoanaugmentedrealitycontentitem.Userselectionofthevisualcode604providestheuserwithaccesstothecorrespondingaugmentedrealitycontentitem.
FIG.7illustratestheuserinterface602inacasewheretheuserselectsthevisualcode604toactivateanaugmentedrealitycontentitem,inaccordancewithsomeexampleembodiments.IntheexampleofFIG.7,theuserselectionisdepictedastouchinput702(e.g.,ataporpressgesture)performedwithrespecttothevisualcode604.Inresponsetothetouchinput702,thevisualcodescannersystem214decodesthemarkingsofthevisualcode(e.g.,toidentifythecorrespondingaugmentedrealitycontentitem),andproceedswithpresentinganimationsdiscussedbelowwithrespecttoFIGS.8,9and10A-10C.
FIG.8illustratestheuserinterface602inwhichthecustomgraphic608ofthevisualcode604isupdatedinresponsetouserselection(e.g.,touchinput702),inaccordancewithsomeexampleembodiments.Theupdatetothecustomgraphic608visuallyindicates,totheuseroftheclientdevice102,receiptofthetouchinput702.AsshownintheexampleofFIG.8,theoriginalcustomgraphic608oftheghosticonisupdatedtowink.Updatingthecustomgraphic608maycorrespondwithreplacingtheoriginalcustomgraphic(e.g.,thenon-winkingghosticon)withthepredefinedstaticimage(e.g.,thewinkingghosticon).Inoneormoreembodiments,updatingthecustomgraphic608correspondswithanimatingthecustomgraphic(e.g.,animatingtheghosticontowinkortoperformanotherpredefinedaction).
FIG.9illustratestheuserinterface602inwhichtheupdatedcustomgraphic608ofFIG.8isanimatedtomovetowardandbeinsertedwithinthecarouselinterface612,inaccordancewithsomeexampleembodiments.IntheexampleofFIG.9,theanimation902representsmovementoftheupdatedcustomgraphic608fromthedisplayareaofthevisualcode604tothedisplayareaofthecarouselinterface612.Thedepictedarrowsoftheanimation902areforillustrativepurposes,toindicatesuchmovement.
WithrespecttoFIG.10A,themessagingclient104updatesthecarouselinterface612toincludeanadditionalicon1002fortheaugmentedrealitycontentitemcorrespondingtothevisualcode604.Forexample,themessagingclient104providesananimationwhichdepictstheadditionalicon1002asbeinginsertedbetweentwoexistingiconsofthecarouselinterface612.Asnotedabove,thecarouselinterface612allowstheusertocyclethroughaugmentedrealitycontentitemsbasedonapredefinedusergesture.WhiletheexampleofFIG.10Aillustratesfouricons,themessagingclient104isconfiguredtopresentadditionaliconsinresponsethepredefinedusergesture(e.g.,aswipegesture).Forexample,inresponsetoaswipegesture,thecarouselinterface612presentsiconsthatarenotpresentlydisplayed,withtheiconsbeingrankedasdescribedabove.
FIGS.10A-10Cillustratedifferentmannersfordownloadingand/oractivatingtheaugmentedrealitycontentitemcorrespondingtothevisualcode604,inaccordancewithsomeexampleembodiments.Asnotedabove,themessagingclient104mayimplementdifferentoptionsfordownloadingthecode(e.g.,assets)fortheaugmentedrealitycontentitem,and/orforactivatingtheaugmentedrealitycontentitem.
FIG.10Aillustratesbothafirstandsecondoption,asdiscussedabovewithrespecttoFIG.5.Inthefirstoption,themessagingclient104requeststhecode(e.g.,assets)fortheaugmentedrealitycontentitem,anddownloadsthecodeinresponsetothefirstuserinputselectingthecode(e.g.,block504ofFIG.5).Inaddition,themessagingclient104activatestheaugmentedrealitycontentiteminresponsetoseconduserinputthatselectstheicon1002insertedinthecarouselinterface612.
Inthesecondoption,themessagingclient104requeststhecode(e.g.,assets)fortheaugmentedrealitycontentitem,downloadsthecodeinresponsetothefirstuserinputselectingthecode,andactivatestheaugmentedrealitycontentiteminresponsetothefirstuserinputselectingthecode(e.g.,block504ofFIG.5).Thus,thesecondoptiondiffersfromthefirstoptioninthatthesecondoptiondoesnotrequiretheseconduserinputthatselectstheicon1002.
FIG.10BillustratesathirdoptionasdiscussedabovewithrespecttoFIG.5.Afterinsertingtheicon1002intothecarouselinterface612,themessagingclient104receivesseconduserinputthatselectstheicon1002.Inresponsetoreceivingtheseconduserinput,themessagingclient104requeststhecode(e.g.,assets)correspondingtotheaugmentedrealitycontentitem,anddownloadsthecode.
AsshownintheexampleofFIG.10B,theicon1002isinitiallygrayedoutwheninsertedintothecarouselinterface612.Userselectionofthegrayedouticoncausesthemessagingclient104torequestanddownloadthecodecorrespondingtotheaugmentedrealitycontentitem.
FIG.10Cdepictsanexpandedviewoftheicon1002,toillustratethemessagingclient104providingananimationshowingadownloadprogressfortheaugmentedrealitycontentitem.Forexample,theanimationprogressivelyremovesthegraymaskfortheicon1002toindicatedownloadprogressandcompletion.
Themessagingclient104displayscapturedimagedatacomprisingavisualcode,thevisualcodeincludingacustomgraphicandbeingdecodabletoaccessafirstfeatureofthemessagingapplication(block1102).Thevisualcodemayincludeauniqueconfigurationofmarkingswhichisdecodabletoaccesstothefirstfeature.Themessagingclient104receivesuserinputselectingthevisualcode(block1104).
Themessagingclient104displaysanupdatedversionofthecustomgraphic(block1106).Theupdatedversionofthecustomgraphicmaycorrespondtoapredefinedstaticimage.Alternatively,theupdatedversionofthecustomgraphicmaycorrespondtoapredefinedanimation.
Themessagingclient104providesananimationwhichdepictstheupdatedversionofthecustomgraphicasmovingfromthevisualcodetoaninterfaceelementcomprisingagroupoficons,eachiconwithinthegroupoficonsbeinguser-selectabletoaccessarespectivesecondfeatureofthemessagingapplication(block1108).
Theinterfaceelementmaybeacarouselinterfacefornavigatingthegroupoficons.Thefirstfeatureandeachoftherespectivesecondfeaturesmaycorrespondtoarespectiveaugmentedrealitycontentitemforapplyingtothecapturedimagedata.
Themessagingclient104maydownload,fromthemessagingserversystem108,codeforafirstaugmentedrealitycontentitemcorrespondingtothefirstfeature.Themessagingclient104mayactivate,basedondownloadingthecode,thefirstaugmentedrealitycontentitemwithrespecttothecapturedimagedata.
Downloadingthecodemaybeinresponsetoseconduserinputselectingtheadditionaliconwithinthegroupoficons.Alternatively,downloadingthecodemaybeinresponsetoreceivingtheuserinput(perblock1104),andactivatingthefirstaugmentedrealitycontentitemmaybeinresponsetoreceivingseconduserinputselectingtheadditionaliconwithinthegroupoficons.
Themessagingclient104updatesthegroupoficonstoincludeanadditionaliconwhichisuser-selectabletoaccessthefirstfeatureofthemessagingapplication(block1110).
FIG.12isaschematicdiagramillustratinganaccess-limitingprocess1200,intermsofwhichaccesstocontent(e.g.,anephemeralmessage1202,andassociatedmultimediapayloadofdata)oracontentcollection(e.g.,anephemeralmessagegroup1204)maybetime-limited(e.g.,madeephemeral).
Anephemeralmessage1202isshowntobeassociatedwithamessagedurationparameter1206,thevalueofwhichdeterminesanamountoftimethattheephemeralmessage1202willbedisplayedtoareceivinguseroftheephemeralmessage1202bythemessagingclient104.Inoneexample,anephemeralmessage1202isviewablebyareceivinguserforuptoamaximumof10seconds,dependingontheamountoftimethatthesendinguserspecifiesusingthemessagedurationparameter1206.
Themessagedurationparameter1206andthemessagereceiveridentifier424areshowntobeinputstoamessagetimer1210,whichisresponsiblefordeterminingtheamountoftimethattheephemeralmessage1202isshowntoaparticularreceivinguseridentifiedbythemessagereceiveridentifier424.Inparticular,theephemeralmessage1202willonlybeshowntotherelevantreceivinguserforatimeperioddeterminedbythevalueofthemessagedurationparameter1206.Themessagetimer1210isshowntoprovideoutputtoamoregeneralizedephemeraltimersystem202,whichisresponsiblefortheoveralltimingofdisplayofcontent(e.g.,anephemeralmessage1202)toareceivinguser.
Theephemeralmessage1202isshowninFIG.12tobeincludedwithinanephemeralmessagegroup1204(e.g.,acollectionofmessagesinapersonalstory,oraneventstory).Theephemeralmessagegroup1204hasanassociatedgroupdurationparameter1208,avalueofwhichdeterminesatimedurationforwhichtheephemeralmessagegroup1204ispresentedandaccessibletousersofthemessagingsystem100.Thegroupdurationparameter1208,forexample,maybethedurationofamusicconcert,wheretheephemeralmessagegroup1204isacollectionofcontentpertainingtothatconcert.Alternatively,auser(eithertheowninguseroracuratoruser)mayspecifythevalueforthegroupdurationparameter1208whenperformingthesetupandcreationoftheephemeralmessagegroup1204.
Additionally,eachephemeralmessage1202withintheephemeralmessagegroup1204hasanassociatedgroupparticipationparameter1212,avalueofwhichdeterminesthedurationoftimeforwhichtheephemeralmessage1202willbeaccessiblewithinthecontextoftheephemeralmessagegroup1204.Accordingly,aparticularephemeralmessagegroup1204may“expire”andbecomeinaccessiblewithinthecontextoftheephemeralmessagegroup1204,priortotheephemeralmessagegroup1204itselfexpiringintermsofthegroupdurationparameter1208.Thegroupdurationparameter1208,groupparticipationparameter1212,andmessagereceiveridentifier424eachprovideinputtoagrouptimer1214,whichoperationallydetermines,firstly,whetheraparticularephemeralmessage1202oftheephemeralmessagegroup1204willbedisplayedtoaparticularreceivinguserand,ifso,forhowlong.Notethattheephemeralmessagegroup1204isalsoawareoftheidentityoftheparticularreceivinguserasaresultofthemessagereceiveridentifier424.
Accordingly,thegrouptimer1214operationallycontrolstheoveralllifespanofanassociatedephemeralmessagegroup1204,aswellasanindividualephemeralmessage1202includedintheephemeralmessagegroup1204.Inoneexample,eachandeveryephemeralmessage1202withintheephemeralmessagegroup1204remainsviewableandaccessibleforatimeperiodspecifiedbythegroupdurationparameter1208.Inafurtherexample,acertainephemeralmessage1202mayexpire,withinthecontextofephemeralmessagegroup1204,basedonagroupparticipationparameter1212.Notethatamessagedurationparameter1206maystilldeterminethedurationoftimeforwhichaparticularephemeralmessage1202isdisplayedtoareceivinguser,evenwithinthecontextoftheephemeralmessagegroup1204.Accordingly,themessagedurationparameter1206determinesthedurationoftimethataparticularephemeralmessage1202isdisplayedtoareceivinguser,regardlessofwhetherthereceivinguserisviewingthatephemeralmessage1202insideoroutsidethecontextofanephemeralmessagegroup1204.
Theephemeraltimersystem202mayfurthermoreoperationallyremoveaparticularephemeralmessage1202fromtheephemeralmessagegroup1204basedonadeterminationthatithasexceededanassociatedgroupparticipationparameter1212.Forexample,whenasendinguserhasestablishedagroupparticipationparameter1212of24hoursfromposting,theephemeraltimersystem202willremovetherelevantephemeralmessage1202fromtheephemeralmessagegroup1204afterthespecified24hours.Theephemeraltimersystem202alsooperatestoremoveanephemeralmessagegroup1204wheneitherthegroupparticipationparameter1212foreachandeveryephemeralmessage1202withintheephemeralmessagegroup1204hasexpired,orwhentheephemeralmessagegroup1204itselfhasexpiredintermsofthegroupdurationparameter1208.
Incertainusecases,acreatorofaparticularephemeralmessagegroup1204mayspecifyanindefinitegroupdurationparameter1208.Inthiscase,theexpirationofthegroupparticipationparameter1212forthelastremainingephemeralmessage1202withintheephemeralmessagegroup1204willdeterminewhentheephemeralmessagegroup1204itselfexpires.Inthiscase,anewephemeralmessage1202,addedtotheephemeralmessagegroup1204,withanewgroupparticipationparameter1212,effectivelyextendsthelifeofanephemeralmessagegroup1204toequalthevalueofthegroupparticipationparameter1212.
Responsivetotheephemeraltimersystem202determiningthatanephemeralmessagegroup1204hasexpired(e.g.,isnolongeraccessible),theephemeraltimersystem202communicateswiththemessagingsystem100(and,forexample,specificallythemessagingclient104)tocauseanindicium(e.g.,anicon)associatedwiththerelevantephemeralmessagegroup1204tonolongerbedisplayedwithinauserinterfaceofthemessagingclient104.Similarly,whentheephemeraltimersystem202determinesthatthemessagedurationparameter1206foraparticularephemeralmessage1202hasexpired,theephemeraltimersystem202causesthemessagingclient104tonolongerdisplayanindicium(e.g.,aniconortextualidentification)associatedwiththeephemeralmessage1202.
Thememory1306includesamainmemory1314,astaticmemory1316,andastorageunit1318,bothaccessibletotheprocessors1304viathebus1340.Themainmemory1306,thestaticmemory1316,andstorageunit1318storetheinstructions1310embodyinganyoneormoreofthemethodologiesorfunctionsdescribedherein.Theinstructions1310mayalsoreside,completelyorpartially,withinthemainmemory1314,withinthestaticmemory1316,withinmachine-readablemedium1320withinthestorageunit1318,withinatleastoneoftheprocessors1304(e.g.,withintheProcessor'scachememory),oranysuitablecombinationthereof,duringexecutionthereofbythemachine1300.
Infurtherexamples,theI/Ocomponents1302mayincludebiometriccomponents1330,motioncomponents1332,environmentalcomponents1334,orpositioncomponents1336,amongawidearrayofothercomponents.Forexample,thebiometriccomponents1330includecomponentstodetectexpressions(e.g.,handexpressions,facialexpressions,vocalexpressions,bodygestures,oreye-tracking),measurebiosignals(e.g.,bloodpressure,heartrate,bodytemperature,perspiration,orbrainwaves),identifyaperson(e.g.,voiceidentification,retinalidentification,facialidentification,fingerprintidentification,orelectroencephalogram-basedidentification),andthelike.Themotioncomponents1332includeaccelerationsensorcomponents(e.g.,accelerometer),gravitationsensorcomponents,rotationsensorcomponents(e.g.,gyroscope).
Theenvironmentalcomponents1334include,forexample,oneorcameras(withstillimage/photographandvideocapabilities),illuminationsensorcomponents(e.g.,photometer),temperaturesensorcomponents(e.g.,oneormorethermometersthatdetectambienttemperature),humiditysensorcomponents,pressuresensorcomponents(e.g.,barometer),acousticsensorcomponents(e.g.,oneormoremicrophonesthatdetectbackgroundnoise),proximitysensorcomponents(e.g.,infraredsensorsthatdetectnearbyobjects),gassensors(e.g.,gasdetectionsensorstodetectionconcentrationsofhazardousgasesforsafetyortomeasurepollutantsintheatmosphere),orothercomponentsthatmayprovideindications,measurements,orsignalscorrespondingtoasurroundingphysicalenvironment.
Withrespecttocameras,theclientdevice102mayhaveacamerasystemcomprising,forexample,frontcamerasonafrontsurfaceoftheclientdevice102andrearcamerasonarearsurfaceoftheclientdevice102.Thefrontcamerasmay,forexample,beusedtocapturestillimagesandvideoofauseroftheclientdevice102(e.g.,“selfies”),whichmaythenbeaugmentedwithaugmentationdata(e.g.,filters)describedabove.Therearcamerasmay,forexample,beusedtocapturestillimagesandvideosinamoretraditionalcameramode,withtheseimagessimilarlybeingaugmentedwithaugmentationdata.Inadditiontofrontandrearcameras,theclientdevice102mayalsoincludea360°cameraforcapturing360°photographsandvideos.
Further,thecamerasystemofaclientdevice102mayincludedualrearcameras(e.g.,aprimarycameraaswellasadepth-sensingcamera),oreventriple,quadorpentarearcameraconfigurationsonthefrontandrearsidesoftheclientdevice102.Thesemultiplecamerassystemsmayincludeawidecamera,anultra-widecamera,atelephotocamera,amacrocameraandadepthsensor,forexample.
Thepositioncomponents1336includelocationsensorcomponents(e.g.,aGPSreceivercomponent),altitudesensorcomponents(e.g.,altimetersorbarometersthatdetectairpressurefromwhichaltitudemaybederived),orientationsensorcomponents(e.g.,magnetometers),andthelike.
Communicationmaybeimplementedusingawidevarietyoftechnologies.TheI/Ocomponents1302furtherincludecommunicationcomponents1338operabletocouplethemachine1300toanetwork1322ordevices1324viarespectivecouplingorconnections.Forexample,thecommunicationcomponents1338mayincludeanetworkinterfaceComponentoranothersuitabledevicetointerfacewiththenetwork1322.Infurtherexamples,thecommunicationcomponents1338mayincludewiredcommunicationcomponents,wirelesscommunicationcomponents,cellularcommunicationcomponents,NearFieldCommunication(NFC)components,Bluetoothcomponents(e.g.,BluetoothLowEnergy),Wi-Ficomponents,andothercommunicationcomponentstoprovidecommunicationviaothermodalities.Thedevices1324maybeanothermachineoranyofawidevarietyofperipheraldevices(e.g.,aperipheraldevicecoupledviaaUSB).
Moreover,thecommunicationcomponents1338maydetectidentifiersorincludecomponentsoperabletodetectidentifiers.Forexample,thecommunicationcomponents1338mayincludeRadioFrequencyIdentification(RFID)tagreadercomponents,NFCsmarttagdetectioncomponents,opticalreadercomponents(e.g.,anopticalsensortodetectone-dimensionalbarcodessuchasUniversalProductCode(UPC)barcode,multi-dimensionalbarcodessuchasQuickResponse(QR)code,Azteccode,DataMatrix,Dataglyph,MaxiCode,PDF417,UltraCode,UCCRSS-2Dbarcode,andotheropticalcodes),oracousticdetectioncomponents(e.g.,microphonestoidentifytaggedaudiosignals).Inaddition,avarietyofinformationmaybederivedviathecommunicationcomponents1338,suchaslocationviaInternetProtocol(IP)geolocation,locationviaWi-Fisignaltriangulation,locationviadetectinganNFCbeaconsignalthatmayindicateaparticularlocation,andsoforth.
Thevariousmemories(e.g.,mainmemory1314,staticmemory1316,andmemoryoftheprocessors1304)andstorageunit1318maystoreoneormoresetsofinstructionsanddatastructures(e.g.,software)embodyingorusedbyanyoneormoreofthemethodologiesorfunctionsdescribedherein.Theseinstructions(e.g.,theinstructions1310),whenexecutedbyprocessors1304,causevariousoperationstoimplementthedisclosedexamples.
Theinstructions1310maybetransmittedorreceivedoverthenetwork1322,usingatransmissionmedium,viaanetworkinterfacedevice(e.g.,anetworkinterfacecomponentincludedinthecommunicationcomponents1338)andusinganyoneofseveralwell-knowntransferprotocols(e.g.,hypertexttransferprotocol(HTTP)).Similarly,theinstructions1310maybetransmittedorreceivedusingatransmissionmediumviaacoupling(e.g.,apeer-to-peercoupling)tothedevices1324.
FIG.14isablockdiagram1400illustratingasoftwarearchitecture1404,whichcanbeinstalledonanyoneormoreofthedevicesdescribedherein.Thesoftwarearchitecture1404issupportedbyhardwaresuchasamachine1402thatincludesprocessors1420,memory1426,andI/Ocomponents1438.Inthisexample,thesoftwarearchitecture1404canbeconceptualizedasastackoflayers,whereeachlayerprovidesaparticularfunctionality.Thesoftwarearchitecture1404includeslayerssuchasanoperatingsystem1412,libraries1410,frameworks1408,andapplications1406.Operationally,theapplications1406invokeAPIcalls1450throughthesoftwarestackandreceivemessages1452inresponsetotheAPIcalls1450.
Theoperatingsystem1412manageshardwareresourcesandprovidescommonservices.Theoperatingsystem1412includes,forexample,akernel1414,services1416,anddrivers1422.Thekernel1414actsasanabstractionlayerbetweenthehardwareandtheothersoftwarelayers.Forexample,thekernel1414providesmemorymanagement,processormanagement(e.g.,scheduling),componentmanagement,networking,andsecuritysettings,amongotherfunctionality.Theservices1416canprovideothercommonservicesfortheothersoftwarelayers.Thedrivers1422areresponsibleforcontrollingorinterfacingwiththeunderlyinghardware.Forinstance,thedrivers1422canincludedisplaydrivers,cameradrivers,BLUETOOTHorBLUETOOTHLowEnergydrivers,flashmemorydrivers,serialcommunicationdrivers(e.g.,USBdrivers),WI-FIdrivers,audiodrivers,powermanagementdrivers,andsoforth.
Theframeworks1408provideacommonhigh-levelinfrastructurethatisusedbytheapplications1406.Forexample,theframeworks1408providevariousgraphicaluserinterface(GUI)functions,high-levelresourcemanagement,andhigh-levellocationservices.Theframeworks1408canprovideabroadspectrumofotherAPIsthatcanbeusedbytheapplications1406,someofwhichmaybespecifictoaparticularoperatingsystemorplatform.
“Carriersignal”referstoanyintangiblemediumthatiscapableofstoring,encoding,orcarryinginstructionsforexecutionbythemachine,andincludesdigitaloranalogcommunicationssignalsorotherintangiblemediatofacilitatecommunicationofsuchinstructions.Instructionsmaybetransmittedorreceivedoveranetworkusingatransmissionmediumviaanetworkinterfacedevice.
“Clientdevice”referstoanymachinethatinterfacestoacommunicationsnetworktoobtainresourcesfromoneormoreserversystemsorotherclientdevices.Aclientdevicemaybe,butisnotlimitedto,amobilephone,desktopcomputer,laptop,portabledigitalassistants(PDAs),smartphones,tablets,ultrabooks,netbooks,laptops,multi-processorsystems,microprocessor-basedorprogrammableconsumerelectronics,gameconsoles,set-topboxes,oranyothercommunicationdevicethatausermayusetoaccessanetwork.
“Computer-readablestoragemedium”referstobothmachine-storagemediaandtransmissionmedia.Thus,thetermsincludebothstoragedevices/mediaandcarrierwaves/modulateddatasignals.Theterms“machine-readablemedium,”“computer-readablemedium”and“device-readablemedium”meanthesamethingandmaybeusedinterchangeablyinthisdisclosure.
“Ephemeralmessage”referstoamessagethatisaccessibleforatime-limitedduration.Anephemeralmessagemaybeatext,animage,avideoandthelike.Theaccesstimefortheephemeralmessagemaybesetbythemessagesender.Alternatively,theaccesstimemaybeadefaultsettingorasettingspecifiedbytherecipient.Regardlessofthesettingtechnique,themessageistransitory.
“Non-transitorycomputer-readablestoragemedium”referstoatangiblemediumthatiscapableofstoring,encoding,orcarryingtheinstructionsforexecutionbyamachine.
“Signalmedium”referstoanyintangiblemediumthatiscapableofstoring,encoding,orcarryingtheinstructionsforexecutionbyamachineandincludesdigitaloranalogcommunicationssignalsorotherintangiblemediatofacilitatecommunicationofsoftwareordata.Theterm“signalmedium”shallbetakentoincludeanyformofamodulateddatasignal,carrierwave,andsoforth.Theterm“modulateddatasignal”meansasignalthathasoneormoreofitscharacteristicssetorchangedinsuchamatterastoencodeinformationinthesignal.Theterms“transmissionmedium”and“signalmedium”meanthesamethingandmaybeusedinterchangeablyinthisdisclosure.